KR20210083997A - Electronic device of vehicle for detecting object and operating method thereof - Google Patents

Abstract

Various embodiments of the present invention relate to a vehicle electronic device for detecting objects, which detects objects by considering the amount of light entering a camera, and an operation method thereof. According to the present invention, the electronic device comprises: at least one light source; at least one camera for detecting surrounding objects during exposure time; and a processor. The processor measures the amount of light entering the camera by using the camera; compares the measured light amount with a predetermined maximum allowable light amount; when the measured light amount is less than or equal to the predetermined maximum allowable light amount, maintains the exposure time of the camera; and when the measured light amount is greater than the predetermined maximum allowable light amount, reduces the exposure time of the camera. At least one of an autonomous vehicle, a user terminal, and a server can be linked with an artificial intelligence module, a drone (unmanned aerial vehicle, UAV), a robot, an augmented reality (AR) device, a virtual reality (VR) device, a device related to a fifth-generation (5G) service, and the like.

Description

An electronic device of a vehicle for detecting an object and an operating method thereof {ELECTRONIC DEVICE OF VEHICLE FOR DETECTING OBJECT AND OPERATING METHOD THEREOF}

Various embodiments of the present disclosure relate to an electronic device of a vehicle for detecting an object and an operating method thereof.

The autonomous driving vehicle refers to a vehicle having a function that can drive itself without a user's manipulation. The autonomous driving vehicle may perform autonomous driving without a user's manipulation based on information obtained from at least one sensor mounted on the vehicle.

The autonomous vehicle secures a field of view by irradiating light rays using a light source included in a headlamp, for example, and detects objects around the vehicle using a lidar sensor and/or a camera sensor included in the vehicle. can The autonomous vehicle may perform automatic driving based on a result of object detection using a lidar sensor and/or a camera sensor.

When a vehicle detects an object using a camera, a whiteout phenomenon may occur due to a nearby light source (eg, a street lamp or a light source included in a headlamp of another vehicle). For example, a white-out phenomenon in which an image of the camera is displayed white may occur because a larger amount of light than a charge-coupled device (CCD) can process is introduced into the vehicle's camera. When the whiteout phenomenon occurs, the vehicle cannot detect a surrounding object, and thus, there is a problem in that the possibility of an accident occurring is increased.

Accordingly, various embodiments of the present disclosure provide an electronic device for a vehicle that detects an object in consideration of the amount of light introduced into a camera, and an operating method thereof.

The technical problems to be achieved in this document are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the art to which the present disclosure belongs from the description below. will be able

According to various embodiments, an electronic device includes at least one light source, at least one camera for detecting an object around the vehicle during an exposure time, and a processor, wherein the processor is configured to communicate with the camera using the camera. Measure the amount of incoming light, compare the measured light amount with a specified maximum allowable amount of light, and if the measured light amount is less than or equal to the specified maximum allowable light amount, maintain the exposure time of the camera, and the measured light amount If it is greater than the specified maximum allowable light amount, the exposure time of the camera may be reduced.

According to various embodiments of the present disclosure, the method of operating an electronic device includes measuring an amount of light introduced into the camera using a camera included in a lamp, comparing the measured amount of light with a specified maximum allowable amount of light, and the measured maintaining the exposure time of the camera when the amount of light is less than or equal to a specified maximum allowable light amount, decreasing the exposure time of the camera when the measured light amount is greater than a specified maximum allowable light amount, the exposure time while using the camera to detect an object around the vehicle.

The electronic device according to various embodiments of the present disclosure may prevent a whiteout phenomenon from occurring by controlling the exposure time of the camera in consideration of the amount of light flowing into the camera inside the headlamp of the vehicle.

An electronic device according to various embodiments of the present disclosure controls the sampling timing of the camera based on the period and intensity of light rays entering the camera inside the headlamp of the vehicle, thereby providing the light source of another vehicle and the own vehicle (own vehicle). It is possible to accurately detect objects around the vehicle by minimizing the influence of the vehicle.

1A illustrates sensors included in a headlamp of a vehicle according to various embodiments of the present disclosure;
1B illustrates a field of view (FOV) of sensors in a headlamp in accordance with various embodiments.
2 is a block diagram of an electronic device for controlling a headlamp according to various embodiments of the present disclosure;
3 is a flowchart of detecting an object using a camera in an electronic device according to various embodiments of the present disclosure;
4 is a flowchart of controlling an image sampling start time of a camera sensor in an electronic device according to various embodiments of the present disclosure;
5A and 5B are exemplary diagrams of changing an image sampling time of a camera sensor based on a period of a light beam of another vehicle in an electronic device according to various embodiments of the present disclosure;
6 is a flowchart of detecting an object based on brightness values of pixels in an image in an electronic device according to various embodiments of the present disclosure;
7 is an exemplary view in which light rays irradiated from a plurality of light sources are introduced into a side camera included in a headlamp of a vehicle according to various embodiments of the present disclosure;
8 is an exemplary diagram for dividing pixel areas corresponding to a plurality of light sources in an electronic device according to various embodiments of the present disclosure;
9 is a flowchart of controlling a sampling time interval according to a reception intensity of a light beam in an electronic device according to various embodiments of the present disclosure;
10 is an exemplary diagram of increasing a sampling time interval according to a reception intensity of a light beam in an electronic device according to various embodiments of the present disclosure;
11 is a flowchart of determining an image sampling period and interval of a camera sensor in an electronic device according to various embodiments of the present disclosure;
12A and 12B are exemplary diagrams of controlling an image sampling period and period of a camera sensor based on a light irradiation period and intensity of a light source of another vehicle in an electronic device according to various embodiments of the present disclosure;
13 is a flowchart of detecting an object using a front camera and a side camera in an electronic device according to various embodiments of the present disclosure;
14A is an exemplary view in which a light beam irradiated from at least one light source is introduced into each of a front camera and a side camera included in a headlamp of a vehicle according to various embodiments of the present disclosure;
14B is an exemplary diagram illustrating an image obtained by photographing a front vehicle using a front camera and a side camera in an electronic device of a vehicle according to various embodiments of the present disclosure;
15 is an exemplary diagram illustrating an image sampling period of a front camera and a side camera of a vehicle according to various embodiments of the present disclosure;

Advantages and features of the present disclosure, and methods of achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below, but will be implemented in various different forms, and only the present embodiments allow the disclosure of the present disclosure to be complete, and are common in the art to which the present disclosure pertains. It is provided to fully inform those with knowledge of the scope of the disclosure, which is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

When one component is referred to as “connected to” or “coupled to” with another component, it means that it is directly connected or coupled to another component or intervening another component. including all cases. On the other hand, when one component is referred to as “directly connected to” or “directly coupled to” with another component, it indicates that another component is not interposed therebetween. “and/or” includes each and every combination of one or more of the recited items.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present disclosure. As used herein, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” refers to the presence of one or more other components, steps, operations and/or elements mentioned. or addition is not excluded.

Although the first, second, etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another.

Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present disclosure. Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used with the meaning commonly understood by those of ordinary skill in the art to which this disclosure belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless clearly defined in particular.

The term 'unit' or 'module' used in this embodiment means software or hardware components such as FPGA or ASIC, and 'unit' or 'module' performs certain roles. However, 'part' or 'module' is not meant to be limited to software or hardware. A 'unit' or 'module' may be configured to reside on an addressable storage medium or may be configured to reproduce one or more processors. Thus, as an example, 'part' or 'module' refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, may include procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. Components and functionality provided in 'units' or 'modules' may be combined into a smaller number of components and 'units' or 'modules' or additional components and 'units' or 'modules' can be further separated.

The steps of a method or algorithm described in connection with some embodiments of the present disclosure may be directly implemented in hardware executed by a processor, a software module, or a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of recording medium known in the art. An exemplary recording medium is coupled to the processor, the processor capable of reading information from, and writing information to, the storage medium. Alternatively, the recording medium may be integral with the processor. The processor and recording medium may reside within an application specific integrated circuit (ASIC). The ASIC may reside within the user terminal.

1A illustrates sensors included in a headlamp of a vehicle according to various embodiments of the present disclosure;

Referring to FIG. 1A , each of the vehicle headlamps 101 and 103 according to various embodiments includes a plurality of cameras 111 , 113 , 131 , 133 , at least one lidar 121 , 123 , It may include at least one light source (141, 143).

According to various embodiments, the first headlamp 101 may include at least one of the first front camera 111 , the first side camera 131 , the first lidar 121 , or the first light source 141 . may include The first headlamp 101 may be a headlamp mounted on the right front side of the vehicle.

The first front camera 111 may be disposed to sense the front of the vehicle, and the first side camera 131 may be disposed to sense at least a portion of the front of the vehicle and a right direction. For example, the first front camera 111 may be disposed on the left side of the interior space of the first headlamp 101 for sensing the front of the vehicle. The first side camera 131 may be disposed on the right side of the interior space of the first headlamp 101 for sensing at least a part of the front of the vehicle and the right direction.

The first lidar 121 may be disposed to sense at least a portion of the front of the vehicle and/or a right direction. For example, the first lidar 121 may be disposed in at least a part of the front of the vehicle and/or in a right area of the interior space area of the first headlamp 101 for right direction sensing. In FIG. 1A , the first lidar 121 is disposed on the first side camera 131 , but this is only an example, and various embodiments of the present disclosure are not limited thereto. For example, the first lidar 121 may be disposed in any one direction of the upper, lower, left, or right side of the first side camera 131 . As another example, the first lidar 121 may be disposed in a central region or a left region of the inner space region of the first head lamp 101 . According to an embodiment, the first lidar 121 may be disposed to be spaced apart from the first side camera 131 or may be disposed to directly contact the first side camera 131 . According to various embodiments, the first head lamp 101 may further include at least one other lidar in addition to the first lidar 121 . For example, the first head lamp 101 may include a plurality of lidars.

The first light source 141 may be disposed in a central area of the inner space area of the first head lamp 101 . This is only an example, and various embodiments of the present disclosure are not limited thereto. For example, the first light source 141 may be disposed in a left area adjacent to the first front camera 111 of the inner space area of the first headlamp 101 or a right area adjacent to the first side camera 131 . can be placed in According to various embodiments, the first head lamp 101 may further include at least one other light source in addition to the first light source 141 . For example, the first head lamp 101 may include a plurality of light sources.

According to various embodiments, the second headlamp 103 includes at least one of the second front camera 113 , the second side camera 133 , the second lidar 123 , or the second light source 143 . may include The second head lamp 1103 may be a head lamp mounted on the left front side of the vehicle.

The second front camera 113 may be disposed to sense the front of the vehicle, and the second side camera 133 may be disposed to sense at least a portion of the front of the vehicle and a left direction. For example, the second front camera 113 may be disposed on the right side of the inner space area of the second head lamp 103 for sensing the front of the vehicle. The second side camera 133 may be disposed in a left area of the interior space area of the second headlamp 103 for sensing at least a part of the front of the vehicle and the left direction.

The second lidar 123 may be disposed to sense at least a portion of the front and/or the right direction of the vehicle. For example, the second lidar 123 may be disposed in at least a portion of the front of the vehicle and in a right area of the inner space area of the second headlamp 103 for sensing in the left direction. In FIG. 1B , the second lidar 123 is disposed on the second side camera 133 , but this is only an example, and various embodiments of the present disclosure are not limited thereto. For example, the second lidar 123 may be disposed in any one direction of the upper, lower, left, or right side of the second side camera 133 . As another example, the second lidar 123 may be disposed in a central region or a right region of the internal space region of the second head lamp 103 . According to one embodiment, the second lidar 123 may be disposed to be spaced apart from the second side camera 133, or may be disposed so as to be in direct contact without being spaced apart. According to various embodiments, the second head lamp 103 may further include at least one other lidar in addition to the second lidar 123 . For example, the second head lamp 103 may include a plurality of lidars.

The second light source 143 may be disposed in a central area of the inner space area of the second head lamp 103 . This is only an example, and various embodiments of the present disclosure are not limited thereto. For example, the second light source 143 is disposed in a right area adjacent to the second front camera 113 among the internal spatial areas of the second head lamp 103 , or in a left area adjacent to the second side camera 133 . can be placed in According to various embodiments, the second head lamp 103 may further include at least one other light source in addition to the second light source 143 . For example, the second head lamp 103 may include a plurality of light sources.

1B illustrates a field of view (FOV) of sensors in a headlamp in accordance with various embodiments. The head lamps 101 and 103 mounted on the vehicle of FIG. 1B may be the head lamps 1101 and 103 illustrated in FIG. 1A .

Referring to FIG. 1B , the vehicle uses the first front camera 111 included in the first headlamp 101 and the second front camera 113 included in the second headlamp 103 to locate an object in front of the vehicle. can be sensed. A field of view of each of the first front camera 111 and the second front camera 113 may be, for example, about 40 degrees.

The vehicle senses an object located in front and/or to the side using the first side camera 131 included in the first headlamp 101 and the second side camera 133 included in the second head lamp 103 . can do. A field of view of each of the first side camera 131 and the second side camera 133 may be, for example, about 140 degrees.

The vehicle senses an object located in the front and/or side by using the first lidar 121 included in the first headlamp 101 and the second lidar 123 included in the second headlamp 103 . can do. A field of view of each of the first lidar 121 and the second lidar 123 may be, for example, about 120 degrees.

The vehicle may secure a view to the front and/or side by using the first light source 141 included in the first headlamp 101 and the second light source 143 included in the second headlamp 103 . have. For example, when driving at night, the vehicle may emit light rays to the first light source 141 and/or the second light source 143 to secure a view forward and/or to the side.

The above-described viewing angles are merely examples, and various embodiments of the present disclosure are not limited thereto. For example, the viewing angles of the sensors 1111 , 1131 , 1121 , 1123 , 1131 , and 1133 included in the first headlamp 101 and the second headlamp 103 may be set to different angles by a designer. have.

In addition, in the above-described FIGS. 1A and 1B and the embodiments to be described later, the two head lamps 101 and 103 located at the front of the vehicle have been described, but various embodiments of the present disclosure are not limited thereto. For example, various embodiments of the present disclosure may be applied in the same manner to headlamps located at the rear of a vehicle.

1A and 1B, the front camera and the side camera are included in one headlamp, but according to various embodiments, the front camera and the side camera may be included in different lamps. For example, the front camera may be included in a headlamp mounted on a front region of the vehicle, and the side camera may be included in a lamp mounted on at least some of the side surfaces of the vehicle.

2 is a block diagram of an electronic device for controlling a headlamp according to various embodiments of the present disclosure; The configuration of the electronic device 200 illustrated in FIG. 2 is an example, and each component may be composed of one chip, component, or electronic circuit, or a combination of chips, components, or electronic circuit. According to another embodiment, some of the components shown in FIG. 2 may be divided into a plurality of components and configured as different chips or components or electronic circuits, and some components may be combined to form a single chip, component, or It may consist of an electronic circuit. According to another embodiment, some of the components shown in FIG. 2 may be omitted or other components not shown in FIG. 2 may be added. According to an embodiment, the components shown in FIG. 2 may be included in the headlamps 101 and 103 of FIGS. 1A and 1B . For example, the electronic device 200 of FIG. 2 may be included in the headlamps 101 and 103 . According to one embodiment, some of the components shown in FIG. 2 are included in the headlamps 101 and 103 , and some other components are not included in the headlamps 101 and 103 and are not included in the autonomous vehicle. By being included in the headlamps 101 and 103 may be electrically connected to some components included in the.

Referring to FIG. 2 , the electronic device 200 may include a processor 210 , a camera module 220 , a light source 230 , and a lidar 240 .

According to various embodiments, the processor 210 may perform an overall control operation for autonomous driving of the vehicle. According to an embodiment, the processor 210 detects (or detects) a surrounding object by controlling at least one of the camera module 220 , the light source, and the lidar 240 , and autonomous driving based on the detection result of the surrounding object can be performed.

According to an embodiment, the processor 210 measures the amount of light flowing into the lens of the at least one front camera 222 and/or the lens of the at least one side camera 224 using the camera module 220 and , an exposure time of at least one camera may be controlled based on the measured amount of light. For example, the processor 210 compares the amount of light entering each lens of the front camera 222 and/or the lens of the side camera 224 with a specified maximum allowable amount of light to the front camera 222 , and/or the side camera. The exposure time of the camera 224 may be reduced or maintained. The amount of light flowing into the lens of the front camera 222 and/or the lens of the at least one side camera 224 is the amount of light emitted from the front lamp of another vehicle and the amount of light emitted from the rear lamp of the other vehicle. , or the amount of light reflected after being irradiated from a front lamp of the own vehicle (hereinafter referred to as 'own vehicle'). The maximum allowable amount of light may be the maximum amount of light that can be processed by a charge-coupled device (CCD) of the camera module 220 . The maximum allowable light amount may be preset by a business operator and/or a designer. For example, the processor 210 may reduce the exposure time of the front camera 222 when the amount of light flowing into the lens of the front camera 222 is greater than a specified maximum allowable light amount. The processor 210 may maintain the exposure time of the front camera 222 when the amount of light flowing into the lens of the front camera 222 is less than or equal to the specified maximum allowable light amount. According to an embodiment, the processor 210 decreases the exposure time of the front camera 222 in stages when the amount of light flowing into the lens of the front camera 222 is gradually increased while satisfying a condition smaller than the specified maximum allowable light amount. can do it

According to an embodiment, the processor 210 controls the brightness of at least one light source during the exposure time of the at least one front camera 222 and/or the at least one side camera 224 included in the camera module 220 . or to control on/off of at least one light source. For example, the processor 210 may decrease the brightness of the side light source by a specified brightness value in order to detect an object located on the side of the vehicle. For example, the processor 210 obtains surrounding environment information including surrounding object information through a communication transceiver (not shown) and/or the camera module 220, and based on the surrounding environment information, an area in which 7 external light sources exist. By predicting the time required to pass by in advance, the brightness of the lateral light source can be adjusted during the predicted time. As another example, the processor 210 turns off a first type of light source (eg, a light-emitting diode (LED)) in order to detect an object located on the side of the vehicle, and a second type of light source (eg: IR LED (infrared LED)) can be turned on. The processor 210 may detect a surrounding object using the second type of light source during the exposure time of the side camera 224 . The communication transceiver may be included in a headlamp or a component included in a vehicle equipped with a headlamp.

According to one embodiment, the processor 210 is configured to receive intensity (or reception level of the light beam) and/or the light beam entering through the at least one front camera 222 , and/or the at least one side camera 224 . one front camera 222 , and/or at least one side camera 224 based on the identified reception intensity of the beam and/or the reception period of the beam ) at which image sampling is performed can be controlled. For example, the processor 210 may be configured for each pixel of an image sensor (eg, a CCD sensor) through at least one front camera 222 and/or at least one side camera 224 included in the camera module 220 . A brightness value may be obtained, and a reception intensity of light entering the at least one front camera 222 and/or the at least one side camera 224 may be determined based on the obtained brightness value. The processor 210 may classify a light source for each light ray based on the reception intensity of the light ray, and may identify a reception period of each light ray based on the divided light source. For example, the processor 210 may distinguish a light beam irradiated from a light source of another vehicle and a light beam reflected by at least one object after being irradiated from a light source of the own vehicle based on the reception intensity of the light beam. The processor 210 may further consider the speed of the vehicle and/or the traveling direction of the vehicle in addition to the reception intensity of the light to distinguish the light sources. The processor 210 may determine and/or change at least one of an image sampling period, a sampling start time, and a sampling time period of an image sensor included in the corresponding camera, based on the reception period and/or the reception intensity of the light beam. According to an embodiment, the image sampling period may be determined and/or changed so as not to overlap with the identified period of reception of the light beam and/or the period of irradiation of the light beam. The irradiation period of the light beam of the other vehicle may correspond to the reception period of the identified light beam. According to an embodiment, the irradiation period of the light beam of the other vehicle may be determined based on a reception period of the identified light beam, the speed of the own vehicle, or the traveling direction of the own vehicle.

According to an embodiment, the processor 210 identifies pixel areas corresponding to each of the plurality of light sources through the sensor of the at least one front camera 222 and/or the at least one side camera 224, A sampling operation may be controlled based on brightness of pixel regions. For example, the processor 210 compares the brightness of each pixel area with the maximum allowable brightness, performs a sampling operation on at least some areas according to the comparison result to detect an object, or stops the sampling operation and the lidar 240 ) to detect an object. For example, the processor 210 checks the brightness of each pixel through the sensor of the side camera 224, and based on the checked brightness of each pixel, the first pixel area into which the first light beam irradiated from the front lamp of the other vehicle 1 is introduced. , a second pixel area into which the second light beam irradiated from the rear lamp of the other vehicle 2 flows, and a third pixel area where the first light beam and the second light beam overlap and enter. When each of the brightness of the first pixel area, the brightness of the second pixel area, and the brightness of the third pixel area is less than or equal to the maximum allowable brightness, the processor 210 may perform a sampling operation to detect the object. When the brightness of some of the brightness of the first pixel area, the brightness of the second pixel area, and the brightness of the third pixel area exceeds the maximum allowable brightness, the processor 210 generates noise in the partial area exceeding the maximum allowable brightness. The object can be detected by processing and performing a sampling operation on another area. When each of the brightness of the first pixel area, the brightness of the second pixel area, and the brightness of the third pixel area exceeds the maximum allowable brightness, the processor 210 may at least temporarily stop the sampling operation. The processor 210 may detect an object using the lidar 240 while the sampling operation is at least temporarily stopped. In order to detect an object using the lidar 240 , the processor 210 may check a direction corresponding to the camera in which the sampling operation is stopped, and rotate the lidar 240 in the confirmed direction. According to one embodiment, when the lidar 240 is rotated, at least one partition wall separating the space from the front camera 222 and/or the side camera 224, or the structure of the light source 230 and the lidar ( The rotation angle of the lidar 240 may be adjusted so that the structure of 240 does not collide.

According to an embodiment, the processor 210 is configured to acquire one image by using at least a portion of raw data sampled a specified number of times (eg, about 60 times) during the entire sampling time period. 220) and detect an object using the acquired image. The entire sampling time interval may be a time interval in which sampling is performed a specified number of times (eg, about 60 times) to acquire one image. The processor 210 may dynamically change the sampling time period in consideration of a change in the amount of light introduced through a sensor of a corresponding camera while acquiring raw data sampled a specified number of times. For example, the processor 210 may measure a reception intensity of a light beam whenever sampling is performed in each sampling period, and may maintain, increase, or decrease the next sampling time period based on the measured reception intensity. For example, the processor 210 measures the reception intensity of the light beam at each sampling time point within the nth sampling interval among the entire sampling time interval, and a change pattern for the reception intensity of the light beam based on the measured reception intensity of the light beam, and/or An average reception strength may be determined. When the change pattern with respect to the reception intensity of the light beam is an increasing pattern within the nth sampling period, the processor 210 may set the n+1th sampling period to be longer than the nth sampling period. If the reception intensity of the light beam within the nth sampling period is not an increasing pattern, but the average reception intensity is greater than the average reception intensity of the n-1th sampling period, the processor 210 sets the n+1th sampling period to the nth sampling period. It can be set longer. When the change pattern with respect to the reception intensity of the light beam within the nth sampling period is a decreasing pattern, the processor 210 may set the n+1th sampling period to be shorter than the nth sampling period. If the reception intensity of the light beam within the nth sampling period is not a decreasing pattern, but the average reception intensity is less than the average reception intensity of the n-1th sampling period, the processor 210 sets the n+1th sampling period to the nth sampling period. It can be set shorter. When the reception intensity of the light beam exceeds the maximum allowable intensity, the processor 210 may not perform sampling on the light beam exceeding the maximum allowable intensity. For example, the processor 210 determines that the object detection accuracy is low due to the excessive amount of light when the reception intensity of the beam exceeds the maximum allowable intensity, and performs sampling while the reception intensity of the beam exceeds the maximum allowable intensity. may not According to an embodiment, the entire sampling time interval may be dynamically changed according to the reception intensity of the light beam.

According to an embodiment, the processor 210 may determine a period (or frequency) at which light is irradiated from a light source of another vehicle, and determine a sampling period and a length of the sampling period based on the confirmed period. For example, when a light beam irradiated from a light source of a front lamp of the other vehicle 1 and a light beam irradiated from a light source of a rear lamp of the other vehicle 2 enter the side camera 224, the processor 210 may Based on the reception intensity, a first period in which light is irradiated from the light source of the front lamp of the other vehicle 1 and a second period in which the light is irradiated from the light source of the rear lamp of the other vehicle 2 may be identified. The processor 210 may determine the sampling period of the side camera 224 and the length of the sampling section so that the sampling period of the side camera 224 does not overlap the first period and the second period.

According to an embodiment, the processor 210 may synthesize images obtained through each of the front camera 222 and the side camera 224 and detect an object using the synthesized image. For example, the processor 210 acquires a first image obtained by photographing the rear of the front vehicle located in the same lane as the own vehicle through the front camera 222 , and the front located in the same lane as the own vehicle through the side camera 224 . A second image obtained by photographing at least a portion of the vehicle and/or at least a portion of another vehicle located in a lane different from that of the own vehicle may be acquired. The processor 210 may obtain an accurate detection result of the vehicle in front by synthesizing the first image and the second image.

According to various embodiments, the camera module 220 may include at least one front camera 222 and at least one side camera 224 . The at least one front camera may include, for example, at least one of the first front camera 111 and the second front camera 113 of FIG. 1 . The at least one side camera 224 may include, for example, the first side camera 131 and the second side camera 133 of FIG. 1 . Each of the front camera 222 and the at least one side camera 224 may acquire at least one image under the control of the processor 210 . According to an embodiment, each of the front camera 222 and the at least one side camera 224 may include at least one lens, at least one image sensor, and at least one image signal processor. The at least one image signal processor may be electrically connected to the image sensor, process a received signal, and generate data about the object based on the processed signal.

The image sensor may acquire an image corresponding to the object by converting light transmitted from the object through at least one lens into an electrical signal. According to an embodiment, each image sensor included in the image sensor may be implemented as, for example, a charged coupled device (CCD) sensor. According to an embodiment, the image sensor may convert a light beam entering through the lens into an electrical signal by performing sampling at a sampling time determined by the processor 210 or the image signal processor. The sampling time of the image sensor may include, for example, at least one of a sampling start time, a sampling period, and a sampling period. According to an embodiment, the image sensor may perform sampling about 60 times in nanosecond units.

According to an embodiment, the image signal processor may perform a control operation on at least one of the components included in the camera module 220 . For example, the image signal processor may perform exposure time control and/or sampling time control for the image sensor. According to an embodiment, the image signal processor may obtain one image by synthesizing raw data obtained through sampling, and may obtain information about the object by analyzing the one obtained image. For example, the image signal processor may obtain at least one of location information of the object, distance information about the object, or relative speed information about the object based on a change in the size of the object in images acquired through the image sensor. can According to an embodiment, the image signal processor may be configured as at least a part of the processor 210 or as a separate processor separate from the processor 210 .

According to various embodiments, the light source 230 may irradiate light at a specified period under the control of the processor 210 . The light source may include, for example, at least one first light source 141 and at least one second light source 143 of FIG. 1 . According to an embodiment, the light source 230 may include a plurality of light sources of different types. For example, the light source 230 may include a light source of a first type, such as a light-emitting diode (LED), and a light source of a second type, such as an infrared LED (IR LED).

According to various embodiments, the lidar 240 may generate information about an object outside the vehicle using laser light. The lidar may include a light transmitter, a light receiver, and at least one LiDAR processor alc. At least one lidar processor may be electrically connected to the light transmitter and the light receiver, process a received signal, and generate data for an object based on the processed signal. The lidar 240 may include, for example, the first lidar 123 and the second lidar 121 of FIG. 1 . The lidar 240 may be implemented in a time of flight (TOF) method or a phase-shift method. Lidar can be implemented as driven or non-driven. When implemented as a drive, the lidar is rotated by a motor and can detect objects around the vehicle. When implemented as a non-driven type, the lidar may detect an object located within a predetermined range with respect to the vehicle by light steering. Vehicle 100 may include a plurality of non-driven lidar. LiDAR detects an object based on a time of flight (TOF) method or a phase-shift method with a laser light medium, and calculates the position of the detected object, the distance to the detected object, and the relative speed. can be obtained

According to an embodiment, the lidar 240 detects an object using a laser beam while sampling of at least one camera included in the camera module 220 is at least temporarily stopped under the control of the processor 210 . can The lidar 240 is rotated to face a first direction corresponding to the camera whose sampling operation is stopped under the control of the processor 210, and emits a laser beam in the first direction to detect an object located in the first direction. can

2 , at least some of the operations of the processor 210 may be performed by the camera module 220 .

According to various embodiments, the electronic device 200 includes at least one light source 230 , at least one camera 220 for detecting an object around the vehicle during an exposure time, and a processor 210 , The processor 210 measures the amount of light flowing into the camera 220 using the camera 220 , compares the measured light amount with a specified maximum allowable light amount, and the measured light amount is a specified maximum allowable light amount If less than or equal to, the exposure time of the camera 220 may be maintained, and when the measured light amount is greater than a specified maximum allowable light amount, the exposure time of the camera 220 may be decreased.

According to an embodiment, the processor may control the brightness of the at least one light source 230 based on the exposure time.

According to an embodiment, the at least one light source 230 includes a light source of a first type and a light source of a second type, and the processor 210 receives the light source of the first type during the exposure time. By turning off the light source and turning on the light source of the second type, it is possible to control the light source of the second type to radiate light to a surrounding object.

According to an embodiment, the processor 210 determines at least one of a reception intensity of a light beam entering the at least one camera 220 or a reception period of the light beam, and the reception intensity of the light beam, or the At least one of a sampling start time, a sampling period, and a sampling period of the image sensor included in the camera may be controlled based on at least one of the light reception periods.

According to an embodiment, the processor 210 determines the sampling start time based on the reception period of the light beam, and the sampling start time is a point in time when the light beam of another vehicle reaches the lens included in the camera. It may be determined at a later time.

According to an embodiment, at the sampling start time, the sampling section of the image sensor includes a direct light receiving section by at least one other light source, a reflected light receiving section by the at least one other light source, or the at least one It may be determined to include at least one of the reception period of the reflected light by the light source.

According to an embodiment, the processor 210 compares the reception intensity of the light beam with a maximum intensity corresponding to the specified maximum allowable light amount, and when the reception intensity of the light beam is less than or equal to the maximum intensity, the camera It is possible to control the image sensor of the to perform sampling during the sampling period based on the sampling start time.

According to an embodiment, when the reception intensity of the light beam is greater than the maximum intensity, the image sensor of the camera may be controlled to stop sampling.

According to an embodiment, the processor 210 may determine a change pattern for the reception intensity of light during the sampling period, and adjust the length of the next sampling period based on the change pattern.

According to an embodiment, the processor 210 determines the average reception intensity of the light beam for each sampling section, determines a change pattern for the average reception intensity of the light beam for each sampling section, and based on the change pattern, The length of the next sampling interval can be adjusted.

According to an embodiment, the processor determines the reception period of the light beam based on the reception intensity of the light beam, determines the light irradiation period of the other vehicle based on the reception period of the light beam, and the light beam of the other vehicle The sampling period may be determined based on the irradiation period.

According to an embodiment, the at least one camera 220 further includes a first camera 222 photographing a first direction, and a second camera 224 photographing a second direction, and the processor ( 220, determines an overlapping area of the first image obtained by using the first camera 222 and the second image obtained by using the second camera 224, and based on the overlapping area, the second The first image and the second image may be synthesized, and an object may be detected using the synthesized image.

According to an embodiment, the processor 210 identifies a plurality of pixel areas corresponding to each of the plurality of light sources based on the brightness value for each pixel of the image sensor included in the camera 220, and compares the brightness value of each of the pixel areas of , with the maximum brightness value, and when the brightness value of each of the plurality of pixel areas is less than or equal to the maximum brightness value, sampling is performed to obtain an image, and the obtained image is can be used to detect objects.

According to an embodiment, it further includes at least one lidar 240 , and the processor 210 is configured to control the camera 220 when a brightness value of each of the plurality of pixel areas is greater than a maximum brightness value. Sampling may be stopped at least temporarily, and the object may be detected using the at least one lidar 240 while the sampling is at least temporarily stopped.

According to an embodiment, the processor 210 may rotate the at least one lidar 240 based on the direction of the camera 220 where the sampling is stopped.

According to an embodiment, when the brightness value of some pixel areas among the plurality of pixel areas is greater than the maximum brightness value, the processor 210 performs noise processing on the partial pixel areas, and performs noise processing on the partial pixel areas except for the partial pixel area. The object may be detected through sampling of pixel regions.

3 is a flowchart of detecting an object using a camera in an electronic device according to various embodiments of the present disclosure; In the following embodiment, each operation may be sequentially performed, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Here, the electronic device may be the electronic device 200 of FIG. 2 .

Referring to FIG. 3 , in operation 302 , the electronic device 200 may measure the amount of light introduced through the camera. According to an embodiment, the processor 210 and/or the camera module 220 of the electronic device 200 may include a lens of at least one front camera 222 and/or a lens of at least one side camera 224 . It is possible to measure the amount of light entering the The amount of light flowing into the lens of the front camera 222 and/or the lens of the at least one side camera 224 is the amount of light emitted from the front lamp of another vehicle and the amount of light emitted from the rear lamp of the other vehicle. , or the amount of light reflected after being irradiated from a front lamp of the own vehicle (hereinafter referred to as 'own vehicle'). For example, the amount of light flowing into the image sensor through the lens of the front camera 222 is the amount of light emitted from the light source of the rear lamp of the other vehicle 1 located in the same lane as the own vehicle, or the amount of light emitted from the front lamp of the own vehicle It may include at least one of the amount of light of the post-reflected light beam. The amount of light flowing into the image sensor through the lens of the side camera 224 is the amount of light emitted from the light source of the rear lamp of the other vehicle 1, and the amount of light emitted from the light source of the rear lamp of the other vehicle 2 located in a different lane from the own vehicle may include at least one of the amount of light of the vehicle, the amount of light emitted from the light source of the front lamp of the other vehicle 3 located in the lane opposite to the lane of the own vehicle, or the amount of light reflected after being irradiated from the front lamp of the own vehicle. . According to an embodiment, the processor 210 and/or the camera module 220 of the electronic device 200 may measure the amount of light flowing into the lens of the camera using an image sensor. For example, the camera module 220 may measure the brightness of each pixel using the CCD sensor, and determine the amount of light flowing into the lens of the camera based on the measured brightness of each pixel.

In operation 304, the electronic device 200 may determine whether the measured light amount is greater than the maximum allowable light amount. The maximum allowable amount of light may be the maximum amount of light that can be processed by an image sensor (eg, a charge-coupled device (CCD) sensor) included in the camera module 220 .

When the measured light amount is greater than the maximum allowable light amount, the electronic device 200 may reduce an exposure time of the camera in operation 306 . For example, when the amount of light introduced through the lens of the front camera 222 is greater than the maximum allowable amount of light, the processor 210 and/or the camera module 220 of the electronic device 200 displays the image of the front camera 222 . By dynamically reducing the exposure time for the sensor, the amount of light introduced through the lens of the front camera 222 may be induced to be smaller than the maximum allowable amount of light.

When the measured light amount is less than or equal to the maximum allowable light amount, the electronic device 200 may maintain the exposure time of the camera in operation 310 . For example, when the amount of light flowing in through the lens of the front camera 222 is less than or equal to the maximum allowable amount of light, the processor 210 and/or the camera module 220 of the electronic device 200 performs the front camera 222 . can maintain the exposure time for the image sensor of

In operation 308, the electronic device 200 may detect an object using a camera. For example, the processor 210 and/or the camera module 220 of the electronic device 200 detects an object using the front camera 222 and/or the side camera 224 for a maintained or reduced exposure time. can do. According to an embodiment, the electronic device 200 may decrease the brightness of at least one light source included in the own vehicle by a specified value so that the amount of light irradiated to the surrounding object decreases during the exposure time. Through this, the electronic device 200 may prevent a white-out phenomenon from occurring in the corresponding camera. According to an embodiment, the electronic device 200 may control on/off of at least one light source during the exposure time. For example, the processor 210 turns off a first type of light source (eg, a light-emitting diode (LED)) to detect an object located on the side of the vehicle, and a second type of light source (eg, an IR LED) (infrared LED) may be turned on The processor 210 may detect a surrounding object using the second type of light source during the exposure time.

In FIG. 3 described above, the exposure time of each of the plurality of cameras included in the vehicle may vary according to the amount of light introduced into the lens of each of the plurality of cameras. For example, the exposure time of the front camera may be maintained as a basic exposure time, and the exposure time of the side camera may be reduced by a time specified by the amount of light flowing into the side camera. According to an embodiment, the exposure time may be adjusted based on the amount of light flowing into the corresponding camera. For example, as the amount of light flowing into the lens of the side camera does not exceed the maximum allowable light amount, but continues to increase in a state greater than the specified threshold light amount, the electronic device 200 continuously increases the exposure time for the image sensor of the side camera. can be reduced to

4 is a flowchart of controlling an image sampling start time of a camera sensor in an electronic device according to various embodiments of the present disclosure; In the following embodiment, each operation may be sequentially performed, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Here, the electronic device may be the electronic device 200 of FIG. 2 . Hereinafter, at least some operations of FIG. 4 will be described with reference to FIGS. 5A and 5B . 5A and 5B are exemplary diagrams of changing an image sampling time of a camera sensor based on a period of a light beam of another vehicle in an electronic device according to various embodiments of the present disclosure; Hereinafter, in FIG. 4 , the first side camera 131 will be described as an example for convenience of description. However, the descriptions of FIG. 4 below may be equally applied to each of the second side camera 133 , the first front camera 111 , and the second front camera 113 .

Referring to FIG. 4 , in operation 402 , the electronic device 200 may check a brightness value for each pixel of the image sensor. For example, the processor 210 and/or the camera module 220 of the electronic device 200 may measure a brightness value for each pixel through an image sensor included in the first side camera 131 .

In operation 404 , the electronic device 200 may determine the reception period of the light beam based on the measured brightness value for each pixel. For example, the processor 210 and/or the camera module 220 may determine a light reception period based on a change in a brightness value for each pixel. The reception period of the light beam may not be constant. For example, as shown in FIG. 5A , even if the other vehicle periodically irradiates (or transmits) light rays according to the designated transmission period 510, the moving direction and/or speed of the other vehicle, the moving direction and/or the own vehicle Depending on the speed, the period 520 at which the light rays reach the lens of the host vehicle may not be constant.

In operation 406 , the electronic device 200 may determine a sampling start time based on the reception period of the light beam. According to one embodiment, the processor 210 and / or the camera module 220, according to the reception period of the light beam, such that the amount of light processed by the image sensor of the first side camera 131 is less than the maximum allowable light, the first A sampling start time of the side camera 131 may be determined. For example, the processor 210 and / or the camera module 220, as shown in Fig. 5a, the light beam transmitted by the other vehicle arrives at the lens of the first side camera 131 of the own vehicle rather than the time point. The image sampling time 530 may be determined such that the first side camera 131 starts sampling at late time points 531 , 533 , 535 , and 537 . According to one embodiment, the processor 210 and/or the camera module 220 determines the image sampling start time point when the light beam of the other vehicle directly reaches the lens of the first side camera 131 (or the light beam of the other vehicle is By setting the time point later than the time point provided to the CCD sensor of the first side camera 131), the image using the light beam directly irradiated from the light source of another vehicle and the light beam directly irradiated from the light source of the other vehicle and then reflected and received sampling can be performed. As another example, as shown in FIG. 5B , the other vehicle moving in the opposite direction to the own vehicle transmits the light beam 1 in the first transmission period 550, and the other vehicle moving in the same direction as the own vehicle transmits the second transmission period ( When the ray 2 is transmitted to 560, the processor 210 and/or the camera module 220 is the point at which the ray 1 of the other vehicle directly reaches the lens of the first side camera 131 (or the ray of the other vehicle is Considering the time point provided to the CCD sensor of the first side camera 131) and the time when the light beam 2 of the other vehicle directly reaches the lens of the first side camera 131, the image sampling start time is dynamically changed (571) , 572, 573, 574) can be done. For example, considering the amount of light of Ray 1 directly irradiated to the lens of the first side camera 131 of the own vehicle, the amount of light received by being reflected, the amount of light of Ray 2 being directly irradiated, and the amount of light of Ray 2 being reflected and received Thus, the sampling start time may be controlled so that the total amount of received light is maintained below the maximum allowable light amount during the sampling time 570 . The sampling start time is at least a portion of a time period in which ray 1 is directly received through the lens of the first lateral camera 131 (direct ray reception time for ray 1), and / or the first lateral camera 131 of It may be determined that at least a part of the time period in which the ray 2 is directly received through the lens (the direct sunlight reception time for the ray 2) overlaps the sampling time period. According to an embodiment, the processor 210 and/or the camera module 220 performs image sampling using light rays (direct rays) directly received from light sources of other vehicles (direct rays) and rays (reflected rays) received after being reflected. can be done Here, the sampling start time of FIGS. 5A and 5B is only an example, and embodiments of the present disclosure are not limited thereto.

6 is a flowchart of detecting an object based on brightness values of pixels in an image in an electronic device according to various embodiments of the present disclosure; In the following embodiment, each operation may be sequentially performed, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Here, the electronic device may be the electronic device 200 of FIG. 2 . Hereinafter, at least some operations of FIG. 6 will be described with reference to FIGS. 7 and 8 . 7 is an exemplary view in which light rays irradiated from a plurality of light sources are introduced into a side camera included in a headlamp of a vehicle according to various embodiments of the present disclosure, and FIG. 8 is a view illustrating a plurality of light sources in an electronic device according to various embodiments of the present disclosure; It is an exemplary diagram for classifying corresponding pixel regions.

Referring to FIG. 6 , in operation 602 , the electronic device 200 may identify pixel areas corresponding to different light sources. According to an embodiment, the processor 210 and/or the camera module 220 of the electronic device 200 measures a brightness value for each pixel through an image sensor, and based on the measured brightness value for each pixel, different light sources pixel areas corresponding to . For example, as shown in FIG. 7 , when the host vehicle 701 travels in the first direction, the light source of the front lamp of the other vehicle 703 travels in the second direction opposite to the first direction. The first light beam 711 and the second light beam 712 by the light source of the rear lamp of the other vehicle 705 located in the same lane may be introduced into the second side camera 133 of the own vehicle 701 at different intervals. have. At this time, the processor 210 and/or the camera module 220 performs the first light beam of the other vehicle 703 among the pixel areas corresponding to the image sensor of the second side camera 133 based on the brightness value for each pixel. The pixel area to which the 711 is irradiated and the pixel area to which the second ray 712 of the other vehicle 705 is irradiated may be identified. According to an embodiment, the processor 210 and/or the camera module 220 determines whether there is a pixel area to which the first ray 711 and the second ray 712 are irradiated together based on the brightness value for each pixel. can be decided For example, as shown in FIG. 8 , the processor 210 and/or the camera module 220 includes pixel areas A1 and 801 to which the first ray 711 of the other vehicle 703 is irradiated, the other vehicle Pixel regions A2 and 803 to which the second ray 712 of 705 are irradiated, and pixel regions A3 and 805 to which the first ray 711 and the second ray 712 are irradiated together may be determined. .

In operation 604 , the electronic device 200 may determine whether a pixel area having a brightness greater than a maximum value among pixel areas corresponding to different light sources exists. For example, the processor 210 and/or the camera module 220 may have greater than the maximum allowable brightness among the three pixel regions A1 801 , A2 803 , and A3 805 as shown in FIG. 8 . It can be determined whether there is a pixel area with brightness.

If there is no pixel area having a brightness greater than the maximum value among the pixel areas corresponding to different light sources, the electronic device 200 performs sampling on each pixel area in operation 614 to detect an object. . For example, the processor 210 and/or the camera module 220 may determine that the brightness of each of the A1 area 801 , the A2 area 803 , and the A3 area 805 as shown in FIG. 8 is higher than the maximum allowable brightness. In the case of less than or equal to, an object may be detected by performing sampling on three pixel areas.

If there is no pixel area having a brightness greater than the maximum value among the pixel areas corresponding to different light sources, the electronic device 200 determines whether the pixel area having a brightness greater than the maximum value is the entire area in operation 606 . can

When the pixel area having a brightness greater than the maximum value is the entire area, the electronic device 200 may stop sampling in operation 608 and detect an object using the lidar in operation 610 . For example, the processor 210 and/or the camera module 220 controls the brightness of each of the three pixel areas A1 area 801 , A2 area 803 , and A3 area 805 as shown in FIG. 8 . is greater than the maximum allowable brightness, sampling is at least temporarily stopped, and the object can be detected using lidar. According to an embodiment, the processor 210 checks a direction corresponding to the camera (eg, the second side camera 133) in which the sampling operation is at least temporarily stopped, and at least one lidar (eg, the second side camera 133) in the confirmed direction. : The second lidar 123) can be rotated.

When the pixel area having a brightness greater than the maximum value is a partial area instead of the entire area, the electronic device 200 performs noise processing on the partial area in operation 616, and performs sampling on other areas except for the partial area of the entire area. object can be detected. For example, the processor 210 and/or the camera module 220 may be configured as an A3 area among the three pixel areas A1 area 801 , A2 area 803 , and A3 area 805 as shown in FIG. 8 . When the brightness of 805 is greater than the maximum allowable brightness, the A3 area 805 may be subjected to noise processing, and the A1 area 801 and the A2 area 803 may be sampled to detect an object.

Hereinafter, in FIGS. 9 to 12B , the electronic device 200 controls the camera module 220 to acquire one image based on raw data obtained by sampling a specified number of times (eg, about 60 times). I will assume and explain.

9 is a flowchart of controlling a sampling time interval according to a reception intensity of a light beam in an electronic device according to various embodiments of the present disclosure; In the following embodiment, each operation may be sequentially performed, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Here, the electronic device may be the electronic device 200 of FIG. 2 . Hereinafter, at least some operations of FIG. 9 will be described with reference to FIG. 10 . 10 is an exemplary diagram of increasing a sampling time interval according to a reception intensity of a light beam in an electronic device according to various embodiments of the present disclosure;

Referring to FIG. 9 , in operation 902 , the electronic device 200 may determine whether a sampling period has arrived. According to an embodiment, the processor 210 and/or the camera module 220 of the electronic device 200 may determine whether a specified sampling period arrives within the entire sampling time period for acquiring one image. For example, as shown in FIG. 10 , it may be determined whether the start time of the first sampling period 1011 has come during the period for sampling one image. According to an embodiment, the sampling period may be determined according to the irradiation period and intensity of the light source of the other vehicle. This will be described in more detail with reference to FIG. 11, which will be described later.

When the sampling period arrives, the electronic device 200 may measure the light reception intensity during the sampling time period in operation 904 . For example, as shown in FIG. 10 , the processor 210 and/or the camera module 220 of the electronic device 200 measures the reception intensity of the light beam using the image sensor during the first sampling period 1011 . can be measured

In operation 906 , the electronic device 200 may determine whether the measured light reception intensity exceeds a specified maximum intensity. The specified maximum intensity may indicate a specified maximum allowable light amount. For example, as shown in FIG. 10 , the processor 210 and/or the camera module 220 of the electronic device 200 determines the reception intensities of the light rays measured during the first sampling period 1011 as the maximum allowable light quantity. can be compared with the maximum intensity representing

When the measured light reception intensity does not exceed the specified maximum intensity, the electronic device 200 may perform sampling in operation 908 . For example, as shown in FIG. 10 , in the processor 210 and/or the camera module 220 of the electronic device 200 , the reception intensities of the light rays measured during the first sampling period 1011 are the maximum allowable light quantity. Since it has a value smaller than the maximum intensity representing , sampling can be performed.

In operation 910, the electronic device 200 may determine whether the intensity of the light beam is gradually increased during the sampling time period.

When the intensity of the light beam is gradually increased during the sampling time period, the electronic device 200 may increase the sampling time period of the next sampling period in operation 918 . The processor 210 and/or the camera module 220 of the electronic device 200 determines that, when the intensity of the light beam is gradually increased during the sampling time period, the distance between the other vehicle irradiating the corresponding light beam and the own vehicle is gradually getting closer. can decide The processor 210 and/or the camera module 220 may increase the sampling time interval of the next sampling period in order to investigate the possibility of a collision as the distance between the other vehicle and the host vehicle gradually decreases. For example, in the processor 210 and/or the camera module 220 of the electronic device 200 , as shown in FIG. 10 , the reception intensities of the light rays measured during the first sampling period 1011 are gradually increased. In this case, the length of the second sampling section 1012 may be increased by a specified time section so that the second sampling section 1012 has a longer time section than the first sampling section 1011 .

If the intensity of the light beam is not gradually increased during the sampling time period, the electronic device 200 may determine whether the intensity of the light beam is gradually decreased during the sampling time period in operation 912 .

When the intensity of the light beam is gradually decreased during the sampling time period, the electronic device 200 may decrease the sampling time period of the next sampling period in operation 914 . The processor 210 and/or the camera module 220 of the electronic device 200 determines that, when the intensity of the light beam is gradually decreased during the sampling time period, the distance between the other vehicle irradiating the light beam and the own vehicle is gradually increasing. can decide The processor 210 and/or the camera module 220 may determine that the probability of a collision is low as the distance between the other vehicle and the host vehicle gradually increases, and may reduce the sampling time interval of the next sampling period.

When the intensity of the light beam is not gradually decreased during the sampling time period, the electronic device 200 may maintain the sampling time period of the next sampling period in operation 920 . The processor 210 and/or the camera module 220 of the electronic device 200 determines that, when the intensity of the light beam is not increased or decreased during the sampling time period, the distance between the other vehicle irradiating the corresponding light beam and the own vehicle is maintained. and the sampling time interval of the next sampling period may be maintained.

When the measured light reception intensity exceeds the specified maximum intensity, the electronic device 200 may stop the sampling operation in operation 916 . For example, the processor 210 and/or the camera module 220 of the electronic device 200 determines that the object detection accuracy is low due to the excessive amount of light when the reception intensity of the light beam exceeds the maximum allowable intensity, Sampling may not be performed while the received intensity of light exceeds the maximum allowable intensity.

Although not shown in the above-described FIG. 9 , the electronic device 200 does not have an increasing pattern in the reception intensity of the light beam within the nth sampling period, but when the average reception intensity is greater than the average reception intensity of the n-1th sampling period, The n+1th sampling time interval may be set longer than the nth sampling time interval. For example, in the processor 210 and/or the camera module 220 of the electronic device 200 , as shown in FIG. 10 , the reception intensities of the light rays measured during the third sampling period 1013 have an increasing pattern. However, when the average reception intensity of the third sampling period 1013 is greater than the average reception intensity of the second sampling period 1012 , the fourth sampling period 1014 has a longer time period than the third sampling period 1013 . The length of the fourth sampling period 1014 may be increased by a specified time period.

As described above, the electronic device 200 may improve object detection accuracy by increasing and/or decreasing the length of the sampling period based on the change pattern of the light reception intensity or the average intensity.

11 is a flowchart of determining an image sampling period and interval of a camera sensor in an electronic device according to various embodiments of the present disclosure; In the following embodiment, each operation may be sequentially performed, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Here, the electronic device may be the electronic device 200 of FIG. 2 . Hereinafter, at least some operations of FIG. 11 will be described with reference to FIGS. 12A and 12B . 12A and 12B are exemplary diagrams of controlling an image sampling period and period of a camera sensor based on a light irradiation period and intensity of a light source of another vehicle in an electronic device according to various embodiments of the present disclosure;

Referring to FIG. 11 , in operation 1102 , the electronic device 200 may identify a period (or frequency) at which light rays from a light source of another vehicle are irradiated. According to an embodiment, the processor 210 of the electronic device 200 and/or the camera module 220 may determine a light irradiation period unique to the other vehicle based on the light reception period of the other vehicle. According to an embodiment, the processor 210 of the electronic device 200, and/or the camera module 220, when a light beam irradiated from a light source included in a lamp of another vehicle flows into at least one camera, the Based on the reception intensity of the beams, a reception period for each beam may be determined. For example, when the light beam irradiated from the light source of the front lamp of the other vehicle 1 and the light beam irradiated from the light source of the rear lamp of the other vehicle 2 flows into the first side camera 131, the light beam is received through the image sensor The intensity may be measured, and the first light beam and the second light beam may be distinguished based on the measured light reception intensity. The processor 210 and/or the camera module 220 may determine the reception period of the first light beam and the reception period of the second light beam by distinguishing the first light beam from the second light beam.

In operation 1104 , the electronic device 200 may determine the sampling period of the own vehicle according to the light irradiation period of the other vehicle. According to an embodiment, the processor 210 and/or the camera module 220 of the electronic device 200 may determine the sampling period so that the sampling operation is not performed during the period in which the light of the other vehicle is irradiated. For example, as shown in FIG. 12A , when another vehicle irradiates a beam with a specified beam irradiation period 1202, the processor 210 of the electronic device 200, and/or the camera module 220 The sampling period 1201 may be determined so that the sampling operation is performed at a point in time when the light of the vehicle is not irradiated. As another example, as shown in FIG. 12B , when each of the other vehicle 1 and the other vehicle 2 irradiates light with a specified light irradiation period 1252 and 1262 , the processor 210 of the electronic device 200, and/or The camera module 220 may determine a sampling period 1241 to perform a sampling operation at a point in time when light is not irradiated from the other vehicle 1 and the other vehicle 2 . In FIGS. 12A and 12B , sampling sections 1231 , 1232 , 1281 , and 1282 are to be determined based on a change pattern of reception intensity or average intensity 1221 , 1222 , 1271 , 1272 in the same manner as described with reference to FIG. can

As described above, the electronic device 200 may control the amount of light to be processed by the image sensor so that the amount of light to be processed by the image sensor does not exceed the maximum allowable amount by determining the sampling period so that the sampling operation is not performed during the period in which light of another vehicle is irradiated. have.

13 is a flowchart of detecting an object using a front camera and a side camera in an electronic device according to various embodiments of the present disclosure; In the following embodiment, each operation may be sequentially performed, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Here, the electronic device may be the electronic device 200 of FIG. 2 . Hereinafter, at least some operations of FIG. 13 will be described with reference to FIGS. 14A, 14B, and 15 . 14A is an exemplary view in which a light beam irradiated from at least one light source is introduced into each of a front camera and a side camera included in a headlamp of a vehicle according to various embodiments, and FIG. 14B is an electronic view of the vehicle according to various embodiments. It is an exemplary view showing an image of a vehicle in front by using a front camera and a side camera in the device. 15 is an exemplary diagram illustrating an image sampling period of a front camera and a side camera of a vehicle according to various embodiments of the present disclosure;

Referring to FIG. 13 , in operation 1302 , the electronic device 200 may check the intensity of a light beam received through each of the front and side cameras. For example, the processor 210 and/or the camera module 220 of the electronic device 200 checks the brightness value for each pixel of the image sensor included in the front camera 222 , and based on the checked brightness value for each pixel Thus, it is possible to determine the reception intensity of the light rays introduced through the front camera 222 . The processor 210 and/or the camera module 220 checks the brightness value for each pixel of the image sensor included in the side camera 224 , and receives a light beam introduced through the side camera based on the checked brightness value for each pixel age can be determined.

In operation 1304, the electronic device 200 may classify the light source according to the reception intensity. For example, the processor 210 and/or the camera module 220 of the electronic device 200 may control the amount of light entering through the front camera 222 based on the reception intensity of the light entering through the front camera 222 . The light source may be divided, and the light source of the light beam introduced through the side camera 224 may be distinguished based on the reception intensity of the light beam introduced through the side camera 224 . For example, the processor 210 and/or the camera module 220 may classify light rays having similar reception intensities into light rays irradiated from the same light source. For example, as shown in FIG. 14A , when the second light beam 1412 enters the front camera 222 of the own vehicle 1401, only the second light beam 1412 reaches the front camera 222, The reception intensity of the light rays measured by the front camera 222 may all be similar. Accordingly, the processor 210 and/or the camera module 220 may identify that the second light beam 1412 irradiated from the other vehicle 1 1402 to the front camera 222 is received. Additionally, as shown in FIG. 14A , the first light beam 1411 and the second light beam 1412 are introduced into the side camera 224 of the own vehicle 1401, and the first light beam arrives at the side camera 224 ( The intensity of 1411 and the intensity of the second ray 1412 may be different. Accordingly, the processor 210 and/or the camera module 220 transmits each light beam to the first beam 1411 irradiated from the second vehicle 1403 based on the reception intensity of the beams measured by the side camera 224 . It can be distinguished whether it corresponds to the second light beam 1412 irradiated from the other vehicle 1 1402 . At this time, the processor 210 and / or the camera module 220 recognizes that the reception intensity of some of the beams reaching the side camera 224 is similar to the reception intensity of the beams reaching the front camera 222, It can be recognized that the beam irradiated from the same light source is introduced into the front camera 222 and the side camera 224 . For example, it may be recognized that the second light beam 1412 irradiated from the light source included in the rear lamp of the other vehicle 1 1402 is introduced into the front camera 222 and the side camera 224 .

In operation 1306, the electronic device 200 may identify a ray irradiation period for each light source. The processor 210 and/or the camera module 220 of the electronic device 200 identifies a ray irradiation period for at least one light source that irradiates a light beam to the front camera 222 , and transmits the light beam to the side camera 224 . It is possible to identify a light irradiation period for at least one light source to be irradiated. An operation of determining a ray irradiation period for at least one light source may be the same as operation 1102 of FIG. 11 .

In operation 1308, the electronic device 200 may control the sampling period and section of the front and side cameras according to the irradiation period and reception intensity of the light beam. The processor 210 and/or the camera module 220 of the electronic device 200 samples the front camera 222 according to the light irradiation period and reception intensity for at least one light source irradiating light to the front camera 222 . Periods and intervals may be determined and/or changed. The processor 210 and/or the camera module 220 determines the sampling period and section of the side camera 224 according to the light irradiation period and reception intensity for at least one light source irradiating light to the side camera 224 and /or can be changed. The operation of determining and/or changing the sampling period and interval may include at least one operation described with reference to FIGS. 9 and 11 . For example, as shown in FIGS. 14A and 15 , the processor 210 and/or the camera module 220 of the rear lamp of the other vehicle 1 1402 irradiates a light beam to the front camera 222 of the own vehicle. Based on the light irradiation period 1512, the CCD sampling period 1544 of the front camera 222 of the own vehicle is determined, and the rear lamp of the other vehicle 1 1402 that irradiates the light with the side camera 224 of the own vehicle. Based on the light irradiation period 1522 and the light irradiation period 1532 of the rear lamp of the second vehicle 1403, the CCD sampling period 1542 of the side camera 224 of the own vehicle may be determined.

In operation 1310 , the electronic device 200 may synthesize images acquired by the front and side cameras. For example, as shown in FIG. 14B , the processor 210 and/or the camera module 220 of the electronic device 200 acquires a first image 1451 obtained by photographing the front of the vehicle using a front camera. and a second image 1452 obtained by photographing the side of the vehicle through the side camera 224 may be acquired. For example, each of the front camera 222 and the side camera 224 performs sampling a specified number of times based on the sampling period and section controlled by the processor 210 and/or the camera module 220, and through sampling A first image or a second image may be obtained using the obtained raw data. The processor 210 and/or the camera module 220 of the electronic device 200 may synthesize the first image and the second image. For example, the processor 210 and/or the camera module 220 searches for an overlapping area or the same object 1402 by comparing the acquired images 1451 and 1452 as shown in FIG. 14B , and the retrieved Two images may be combined based on the object 1402 .

In operation 1312, the electronic device 200 may detect an object using the synthesized image. For example, the processor 210 and/or the camera module 220 of the electronic device 200 uses the synthesized image to provide identification information about an object, location information of an object, distance information about an object, or information about an object. At least one of relative speed information may be obtained.

In the above description, the operations of FIGS. 3 , 4 , 6 , 9 , 11 , and 13 may be performed independently of each other in the electronic device 200 or performed in parallel with each other in relation to each other could be For example, the electronic device 200 may perform FIGS. 9 and 11 in parallel so that at least some of the operations of FIG. 9 and at least some of the operations of FIG. 11 are performed at the same time. have. For example, the electronic device 200 may determine a sampling period using at least some operations of FIG. 9 while determining a sampling period using at least some operations of FIG. 11 .

In various embodiments of the present disclosure described above, a headlamp of a vehicle has been described, but various embodiments of the present disclosure described above are not limited to the headlamp. For example, various embodiments of the present disclosure described above may be equally applied to a side lamp, and/or a rear lamp of a vehicle.

The headlamp and/or the electronic device 200 for controlling the headlamp according to various embodiments of the present disclosure may be mounted on an autonomous vehicle.

Autonomous vehicles are linked to artificial intelligence (Artificial Intelligence) modules, drones (Unmanned Aerial Vehicle, UAV), robots, augmented reality (AR) devices, virtual reality (VR), devices related to 5G services, etc. Or it can be fused. For example, the autonomous driving vehicle may operate in connection with at least one artificial intelligence module or robot included in the vehicle.

The vehicle may, for example, interact with at least one robot. The robot may be an Autonomous Mobile Robot (AMR) capable of driving by itself. The mobile robot is free to move because it can move by itself, and is provided with a plurality of sensors for avoiding obstacles while driving, so that it can run while avoiding obstacles. The mobile robot may be a flying robot (eg, a drone) having a flying device. The mobile robot may be a wheel-type robot having at least one wheel and moving through rotation of the wheel. The mobile robot may be a legged robot having at least one leg and moving using the leg.

The robot can function as a device that complements the convenience of vehicle users. For example, the robot may perform a function of moving a load loaded in a vehicle to a final destination of a user. For example, the robot may perform a function of guiding a user who got off the vehicle to a final destination. For example, the robot may perform a function of transporting a user who got out of a vehicle to a final destination.

At least one electronic device included in the vehicle may communicate with the robot through the communication device. At least one electronic device included in the vehicle may provide data processed by the at least one electronic device included in the vehicle to the robot. For example, the at least one electronic device included in the vehicle may include at least one of object data indicating objects around the vehicle, map data, vehicle state data, vehicle location data, and driving plan data. Either one can be provided to the robot.

At least one electronic device included in the vehicle may receive, from the robot, data processed by the robot. At least one electronic device included in the vehicle may generate a control signal based on data received from the robot. For example, the at least one electronic device included in the vehicle compares and compares information on an object generated by an object detection device (eg, a camera or lidar) with information on an object generated by a robot. Based on the result, a control signal may be generated. At least one electronic device included in the vehicle may generate a control signal to prevent interference between the movement path of the vehicle and the movement path of the robot.

At least one electronic device included in the vehicle may include a software module or a hardware module (hereinafter, referred to as an artificial intelligence module) for implementing artificial intelligence (AI). At least one electronic device included in the vehicle may input acquired data to an artificial intelligence module and use data output from the artificial intelligence module.

The artificial intelligence module may perform machine learning on input data using at least one artificial neural network (ANN). The artificial intelligence module may output driving plan data through machine learning on input data.

At least one electronic device included in the vehicle may generate a control signal based on data output from the artificial intelligence module.

According to an embodiment, at least one electronic device included in a vehicle may receive data processed by artificial intelligence from an external device through a communication device. At least one electronic device included in the vehicle may generate a control signal based on data processed by artificial intelligence.

101: first headlamp 103: second headlamp
111: first front camera 113: second front camera
121: first lidar 123: second lidar
131: first side camera 133: second side camera
141: first light source 143: second light source

Claims (20)

In an electronic device,
at least one light source;
at least one camera for detecting surrounding objects during the exposure time; and
A processor comprising:
Measuring the amount of light entering the camera using the camera,
Comparing the measured light amount with the specified maximum allowable light amount,
If the measured light amount is less than or equal to the specified maximum allowable light amount, maintaining the exposure time of the camera,
The electronic device decreases the exposure time of the camera when the measured light amount is greater than a specified maximum allowable light amount.
According to claim 1,
The processor controls the brightness of the at least one light source based on the exposure time.
The method of claim 1,
The at least one light source includes a light source of a first type and a light source of a second type,
The processor is configured to turn off the first type of light source and turn on the second type of light source during the exposure time to control the second type of light source to irradiate light to a surrounding object.
According to claim 1,
The processor determines at least one of a reception intensity of a light beam entering the at least one camera, or a reception period of the light beam,
An electronic device for controlling at least one of a sampling start time, a sampling period, and a sampling period of the image sensor included in the camera, based on at least one of a reception intensity of the light beam and a reception period of the light beam.
5. The method of claim 4,
The processor determines the sampling start time based on the reception period of the light beam,
The sampling start time is an electronic device that is determined to be later than a point in time when a light beam of another vehicle arrives at a lens included in the camera.
6. The method of claim 5,
The sampling start time is a sampling section of the image sensor, a direct light receiving section by at least one other light source, a reflected light receiving section by the at least one other light source, or a reflected light receiving by the at least one light source The electronic device is determined to include at least one of the sections.
5. The method of claim 4,
The processor compares the received intensity of the light beam with a maximum intensity corresponding to the specified maximum allowable light amount,
When the reception intensity of the light beam is less than or equal to the maximum intensity, the electronic device controls the image sensor of the camera to perform sampling during the sampling period based on the sampling start time.
8. The method of claim 7,
When the received intensity of the light beam is greater than the maximum intensity, the electronic device controls the image sensor of the camera to stop sampling.
8. The method of claim 7,
The processor determines a change pattern for the reception intensity of the light beam during the sampling period,
An electronic device for adjusting a length of a next sampling section based on the change pattern.
8. The method of claim 7,
The processor determines the average reception intensity of the light beam for each sampling section,
Determining a change pattern for the average reception intensity of the light beam for each sampling section,
An electronic device for adjusting a length of a next sampling section based on the change pattern.
5. The method of claim 4,
The processor determines a reception period of the light beam based on the reception intensity of the light beam,
Determine the light irradiation period of the other vehicle based on the receiving period of the light beam,
An electronic device for determining the sampling period based on a light irradiation period of the other vehicle.
5. The method of claim 4,
The at least one camera further includes a first camera for photographing a first direction, and a second camera for photographing a second direction,
The processor determines an overlapping region of a first image acquired using the first camera and a second image acquired using the second camera,
synthesizing the first image and the second image based on the overlapping region,
An electronic device for detecting an object using the synthesized image.
According to claim 1,
The processor identifies a plurality of pixel areas corresponding to each of a plurality of light sources based on a brightness value for each pixel of an image sensor included in the camera,
comparing a brightness value of each of the plurality of pixel areas with a maximum brightness value;
When the brightness value of each of the plurality of pixel areas is less than or equal to the maximum brightness value, sampling is performed to obtain an image,
An electronic device for detecting an object by using the acquired image.
14. The method of claim 13,
It further comprises at least one lidar,
the processor, when the brightness value of each of the plurality of pixel areas is greater than the maximum brightness value, at least temporarily stops sampling of the camera;
The electronic device detects the object using the at least one lidar while the sampling is at least temporarily stopped.
15. The method of claim 14,
The processor rotates the at least one lidar based on a direction of the camera in which the sampling is stopped.
14. The method of claim 13,
the processor, when a brightness value of a partial pixel area among the plurality of pixel areas is greater than a maximum brightness value, performs noise processing on the partial pixel area;
The electronic device detects the object through sampling of pixel areas other than the partial pixel area.
The method of claim 1,
The electronic device is an electronic device mounted on an autonomous vehicle.
A method of operating an electronic device, comprising:
measuring the amount of light introduced into the camera by using a camera included in the lamp;
comparing the measured light amount with a specified maximum allowable light amount;
maintaining the exposure time of the camera when the measured light amount is less than or equal to a specified maximum allowable light amount;
reducing the exposure time of the camera when the measured light amount is greater than a specified maximum allowable light amount;
and detecting an object around the vehicle using the camera during the exposure time.
19. The method of claim 18,
The method further comprising controlling the brightness of at least one light source included in the lamp based on the exposure time.
19. The method of claim 18,
determining at least one of a reception intensity of a light beam entering the at least one camera and a reception period of the light beam; and
The method further comprising controlling at least one of a sampling start time, a sampling period, and a sampling period of the image sensor included in the camera, based on at least one of the reception intensity of the light beam and the reception period of the light beam.

Images