KR102308658B1 - Method of operating an infrared radiation LED for monitoring movement of driver, and driver monitoring system including the infrared radiation LED - Google Patents

Abstract

The present invention discloses a method to monitor a driver by a camera control unit controlling operation of an infrared LED. Specifically, the method comprises the following steps of: transmitting a booting signal to a camera when electric power is entered into the camera control unit through a power unit; transmitting a first trigger signal to activate the camera and transmitting a second trigger signal to activate the infrared LED; receiving video of the driver from the camera; and monitoring condition of the driver by analyzing the video.

Description

Method of operating an infrared radiation LED for monitoring movement of driver, and driver monitoring system including the infrared radiation LED

The present invention relates to a method of operating an infrared LED (Lighting Emitting Diode) for monitoring a driver's movement and a driver monitoring system including the infrared LED, and more particularly, assisting a camera photographing the driver in the driver monitoring system. To a method for controlling the operating period of an infrared LED for

As autonomous vehicles evolve from level 3 to level 4 or level 5, the importance of technology that monitors drivers increases.

In this case, various sensors such as a camera or radar are used to monitor the driver.

Specifically, when monitoring a driver using a camera, the driver can be monitored by illuminating the driver with an infrared LED and receiving only an infrared light source in a wavelength range of 850 nm to 950 nm to the camera in order to develop to be robust to an external light source.

However, when the infrared light source continuously operates, there is a problem in that the temperature of the infrared LED rapidly rises.

In order to solve this problem, the heat problem is solved by receiving an operation signal from the camera and operating the infrared LED only at the moment the camera takes a picture of the driver.

Therefore, in the present invention, an ECU or a microcomputer transmits a trigger signal to a camera to propose a technique for operating an infrared LED.

Korean Patent Laid-Open No. 10-2020-0092739 (Title: Driver condition monitoring method and device, 2020.08.04)

An object of the present invention is to provide a method of operating an infrared LED (Lighting Emitting Diode) for monitoring a driver's movement and a driver monitoring system including the infrared LED.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description. will be able

In the method for the camera control unit to monitor the driver by controlling the operation of the infrared LED according to an embodiment of the present invention, when power is input to the camera control unit through the power supply unit, a booting signal is transmitted to the camera and the camera is operated. Transmitting a first trigger signal and transmitting a second trigger signal for operating the infrared LED, receiving an image of the driver from the camera, and analyzing the captured image, may include monitoring the status of

In this case, the method may further include controlling a transmission period of the second trigger signal based on the analysis result of the photographing.

In addition, the method may further include measuring the temperature of the infrared LED, and controlling a transmission period of the second trigger signal based on the temperature of the infrared LED.

In addition, the method may further include receiving a shooting standby signal for shooting standby from the camera.

The method may further include outputting a warning sound based on the driver's state.

According to the present invention, the camera speed and the infrared LED can be controlled according to the temperature of the infrared LED and the monitoring algorithm, so that the temperature management of the infrared LED is easy and the performance of the monitoring algorithm can be improved.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description. will be.

Other aspects, features and benefits as described above of certain preferred embodiments of the present invention will become more apparent from the following description taken in conjunction with the accompanying drawings.
1 is a view for explaining the configuration of a driver monitoring apparatus according to an embodiment of the present invention.
2 is a view for explaining an operation process of a driver monitoring system according to an embodiment of the present invention.
3 is a diagram for explaining a method of determining a transmission period of a trigger signal for an infrared LED according to an embodiment of the present invention.
4 is a diagram for explaining a method of determining a transmission period of a trigger signal for an infrared LED according to an embodiment of the present invention.
5 is a diagram for explaining a method of determining a transmission period of a trigger signal for an infrared LED according to an embodiment of the present invention.
It should be noted that throughout the drawings, like reference numerals are used to denote the same or similar elements, features, and structures.

In order to make the characteristics and advantages of the problem solving means of the present invention more clear, the present invention will be described in more detail with reference to a specific embodiment of the present invention shown in the accompanying drawings.

However, detailed descriptions of well-known functions or configurations that may obscure the gist of the present invention in the following description and accompanying drawings will be omitted. Also, it should be noted that throughout the drawings, the same components are denoted by the same reference numerals as much as possible.

The terms or words used in the following description and drawings should not be construed as being limited to conventional or dictionary meanings, and the inventor may appropriately define the concept of terms for describing his invention in the best way. It should be interpreted as meaning and concept consistent with the technical idea of the present invention based on the principle that there is. Therefore, the configuration shown in one embodiment and the drawings described in the present specification is only one of the most preferred embodiments of the present invention, and does not represent all of the technical spirit of the present invention, so they can be replaced at the time of the present application. It should be understood that there may be various equivalents and variations that exist.

In addition, terms including ordinal numbers such as first, second, etc. are used to describe various components, and are used only for the purpose of distinguishing one component from other components, and to limit the components. not used For example, without departing from the scope of the present invention, the second component may be referred to as the first component, and similarly, the first component may also be referred to as the second component.

In addition, the terminology used herein is used only to describe a specific embodiment, and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In addition, terms such as "comprises" or "have" described herein are intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, but one or the It should be understood that the above does not preclude the possibility of the existence or addition of other features or numbers, steps, operations, components, parts, or combinations thereof.

In addition, terms such as "unit", "group", and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software or a combination of hardware and software. Also, "a or an", "one", "the" and like related terms are used differently herein in the context of describing the invention (especially in the context of the claims that follow). Unless indicated or clearly contradicted by context, it may be used in a sense including both the singular and the plural.

In addition to the above-mentioned terms, specific terms used in the following description are provided to help the understanding of the present invention, and the use of these specific terms may be changed to other forms without departing from the technical spirit of the present invention.

Furthermore, one embodiment within the scope of the present invention includes computer-readable media having or carrying computer-executable instructions or data structures stored thereon. Such computer readable media can be any available media that can be accessed by a general purpose or special purpose computer system. By way of example, such computer-readable media may be in the form of RAM, ROM, EPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, or computer-executable instructions, computer-readable instructions, or data structures. It may include, but is not limited to, a physical storage medium such as any other medium that can be used to store or convey any program code means in .

Furthermore, the present invention relates to personal computers, laptop computers, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile telephones, PDAs, pagers. It may be implemented in a network computing environment having various types of computer system configurations, including (pager) and the like. The invention may also be practiced in distributed system environments where both local and remote computer systems linked through a network by a wired data link, a wireless data link, or a combination of wired and wireless data links perform tasks. In a distributed system environment, program modules may be located in local and remote memory storage devices.

In addition, it will be understood that each block of the flowchart diagrams and combinations of the flowchart diagrams may be performed by computer program instructions. These computer program instructions may be embodied in a processor of a general purpose computer, special purpose computer, or other programmable data processing equipment, such that the instructions performed by the processor of the computer or other programmable data processing equipment are not described in the flowchart block(s). It creates a means to perform functions. These computer program instructions may also be stored in a computer-usable or computer-readable memory that may direct a computer or other programmable data processing equipment to implement a function in a particular manner, and thus the computer-usable or computer-readable memory. It is also possible that the instructions stored in the flow chart block(s) produce an article of manufacture containing instruction means for performing the function described in the flowchart block(s). The computer program instructions may also be mounted on a computer or other programmable data processing equipment, such that a series of operational steps are performed on the computer or other programmable data processing equipment to create a computer-executed process to create a computer or other programmable data processing equipment. It is also possible that instructions for performing the processing equipment provide steps for performing the functions described in the flowchart block(s).

Additionally, each block may represent a module, segment, or portion of code that includes one or more executable instructions for executing specified logical function(s). It should also be noted that in some alternative implementations it is also possible for the functions recited in blocks to occur out of order. For example, two blocks shown one after another may in fact be performed substantially simultaneously, or it is possible that the blocks are sometimes performed in the reverse order according to the corresponding function.

In describing the embodiments of the present disclosure in detail, an example of a specific system will be mainly targeted, but the main subject matter to be claimed in this specification is to extend the scope disclosed herein to other communication systems and services having a similar technical background. It can be applied within a range that does not deviate significantly, and this will be possible at the discretion of a person with technical knowledge skilled in the art.

Now, let's take a look at the driver monitoring system according to an embodiment of the present invention in earnest.

1 is a block diagram illustrating the configuration of a driver monitoring apparatus for implementing the above-described exemplary embodiment of the present invention.

Referring to FIG. 1 , the driver monitoring device includes a control unit 100 , a camera unit 200 , an infrared LED unit 300 , a storage unit 400 , an input unit 500 , an output unit 600 , and a communication unit 700 , etc. may include.

The camera unit 200 is for capturing an image for monitoring the driver's condition. For example, the camera unit 200 may include at least one camera and/or at least one sensor installed in a driver's seat in a vehicle (vehicle), and the at least one camera and/or at least one sensor may be installed in the vehicle. It can be installed at an angle and/or position to photograph the driver of the (vehicle). As another example, the camera unit 200 may photograph the driver's state through a camera installed on the front and/or side of the driver and transmit this image to the controller 100 .

The infrared LED unit 300 is a device for irradiating infrared rays through the infrared LED in order for the controller 100 to more accurately analyze the image captured by the camera unit 200 . The infrared LED unit 300 may irradiate a light source based on a trigger signal transmitted from the control unit 100 .

The storage unit 400 is a device for storing data, includes a main memory device and an auxiliary memory device, and stores an application program required for a functional operation of the driver monitoring device. The storage unit 400 may largely include a program area and a data area. Here, when each function is activated in response to a user's request, the driver monitoring apparatus provides each function by executing corresponding application programs under the control of the controller 100 .

The input unit 500 receives various information such as number and character information, and transmits input signals related to setting various functions and controlling functions of the driver monitoring apparatus to the control unit 100 . Also, the input unit 500 may include at least one of a keypad and a touchpad that generate an input signal according to a user's touch or manipulation. In this case, the input unit 500 may be configured in the form of a single touch panel (or touch screen) together with the output unit 600 to simultaneously perform input and display functions. In addition, as the input unit 500 , all types of input means that may be developed in the future may be used in addition to input devices such as a keyboard, a keypad, a mouse, and a joystick. In particular, the input unit 500 according to the present invention detects input information input from the user and transmits it to the control unit 100 .

The output unit 600 displays information about a series of operation states and operation results that occur while the function of the driver monitoring device is performed. Also, the output unit 600 may display a menu of the driver monitoring device and user data input by the user. Here, the output unit 600 is a liquid crystal display (LCD), an ultra-thin liquid crystal display (TFT-LCD, Thin Film Transistor LCD), a light emitting diode (LED, Light Emitting Diode), an organic light emitting diode (OLED, Organic LED), an active organic light emitting diode (AMOLED, Active Matrix OLED), a retina display (Retina Display), a flexible display (Flexible display), a three-dimensional (3 Dimension) display, etc. may be configured. In this case, when the output unit 600 is configured in the form of a touch screen, the output unit 600 may perform some or all of the functions of the input unit 500 .

The communication unit 700 may transmit/receive data to and from at least one external device (eg, a black box or a navigation system) through a communication network. In addition, the communication unit 700 includes an RF transmitting unit for up-converting and amplifying the frequency of the transmitted signal, an RF receiving unit for low-noise amplifying a received signal and down-converting the frequency, and data for processing a communication protocol according to a specific communication method processing means and the like. The communication unit 700 may include at least one of a wireless communication module (not shown) and a wired communication module (not shown). In addition, the wireless communication module is a configuration for transmitting and receiving data according to a wireless communication method, and when the driver monitoring device uses wireless communication, data using any one of a wireless network communication module, a wireless LAN communication module, and a wireless fan communication module may be transmitted/received to and from at least one external device. Here, the communication unit 700 may include a plurality of communication modules. When a plurality of communication modules are included in the communication unit 700 , one communication module may perform personal area network (PAN) communication including Bluetooth.

In addition, the other communication module communicates with at least one external device through a communication network. Here, the other communication module may use a wireless communication method such as Wireless LAN (WLAN), Wi-Fi, Wibro, WiMAX, and High Speed Downlink Packet Access (HSDPA).

The control unit 100 may be a process device for driving an operating system (OS) and each component.

Accordingly, the control unit 100 of the driver monitoring device controls to transmit the signal received through the input unit 500 to at least one external device through the communication unit 700, and the beacon, Wi-Fi, It is possible to control to expose the base station signal or information transmitted from at least one external device through the output unit 600 .

Also, the controller 100 of the driver monitoring apparatus may control the overall operation process of the driver monitoring apparatus according to an embodiment of the present invention. In other words, the overall operation process of the driver monitoring apparatus according to an embodiment of the present invention, which will be described later, may be controlled by the controller 100 .

Meanwhile, the control unit 100 may include a power supply unit 110 , a photographing standby unit 120 , a trigger condition monitoring unit 130 , a trigger signal generation unit 140 , and an image analysis unit 150 .

Now, with reference to FIGS. 2 to 4 , the operation of each component of the control unit 100 and/or the overall operation process of the driver monitoring device controlled by the control unit 100 will be described.

First, referring to FIG. 2 , when power for operating the driver monitoring device is input through the power supply unit 110 , the power unit 110 transmits a booting signal for operating the camera to the camera through the camera unit 200 . There is (S205). Upon receiving the booting signal, the camera prepares for an operation for photographing the driver, and when the operation preparation is completed, a P clock (Clock; clk) standby signal is generated and transmitted to the photographing standby unit 120 , and the photographing standby unit (120) may receive the standby signal (S210). The shooting standby unit 120 that has received the standby signal receives an image from the camera through the camera unit 200 or performs an operation for controlling the shooting direction, shooting time, shooting interval, etc. of the camera or transmits a command. The entire operation of the controller 100 may be on standby.

The trigger signal generating unit 140 generates a first trigger signal for starting the shooting of the camera and a second trigger signal for operating the infrared LED to assist the camera in shooting, and transmits the first trigger signal to the camera unit ( 200), and a second trigger signal may be transmitted to the infrared LED through the infrared LED unit 300 (S210 to S220).

The camera that receives the first trigger signal starts shooting the driver, and the infrared LED that receives the second trigger signal emits infrared rays. At this time, the infrared LED operates for a predetermined time after receiving the second trigger signal, and if the second trigger signal is not received again within the predetermined time or after a predetermined time has elapsed, the infrared LED may not operate.

That is, the second trigger signal may be transmitted periodically. In this case, a method of determining the period at which the second trigger signal is transmitted will be described in detail later.

The image analysis unit 150 may receive an image of the driver captured by the camera through the camera unit 200 ( S225 ), and may analyze the driver's condition. At this time, if it is determined that the driver's state is in a state that may threaten safe driving, such as drowsy driving or negligence of forward gaze, it is possible to control to transmit a warning sound notification corresponding to each state (S230).

On the other hand, the trigger condition monitoring unit 130 may determine the transmission period of the above-described second trigger signal. That is, in order to reduce the heating phenomenon of the infrared LED, the period of the second trigger signal may be appropriately adjusted. When the trigger condition monitoring unit 130 determines the transmission period of the second trigger signal, the trigger signal generator 140 generates a second trigger signal according to the determined transmission period, and generates a second trigger signal according to the determined transmission period. It may transmit to the infrared LED through the infrared LED unit 300 .

Hereinafter, a method for the trigger condition monitoring unit 130 to determine the transmission period of the second trigger signal will be described.

first embodiment

Referring to FIG. 3 , the trigger condition monitoring unit 130 may receive temperature information of the infrared LED through the infrared LED unit 300 ( S305 ).

For example, based on the temperature information, in the case of identifying the current temperature of the infrared LED, and the current temperature of the infrared LED exceeds the first threshold, it is possible to two-fold increase n ₁ the transmission period of the trigger signal. In this case, n ₁ may be a positive number.

When the transmission period of the second trigger signal is increased, the number of times per time that the second trigger signal is transmitted to the infrared LED is reduced, so the time period during which the operation of the infrared LED is stopped becomes longer, so that the heating phenomenon of the infrared LED can be reduced. .

If the current temperature of the infrared LED is less than or equal to the first threshold, the second trigger signal may be transmitted according to the currently set transmission period without increasing or decreasing the transmission period of the second trigger signal (S310 to S315) .

On the other hand, even when there is a great need to monitor the driver's condition because the temperature of the infrared LED exceeds the first threshold, if the transmission period of the second trigger signal is increased, the driver monitoring device performs the operation necessary for autonomous driving. There may be problems that cannot be done.

Therefore, by adjusting the transmission period of the second trigger signal in advance before the temperature of the infrared LED exceeds the first threshold, there is a method that can delay the time when the temperature of the infrared LED does not reach or reaches the first threshold as much as possible. necessary.

To this end, a first temperature sensor capable of measuring the temperature of the infrared LED and a second temperature sensor capable of measuring the temperature outside the infrared LED are attached to the infrared LED or the infrared LED unit 300 to utilize these temperature sensors. have.

The first temperature sensor measures the current heating temperature of the infrared LED. In addition, the second temperature sensor may be used to measure an external temperature that may affect the heat generation of the infrared LED. Then, the trigger condition monitoring unit 130 extracts the expected rise temperature curve of the temperature of the infrared LED for a certain period of time, such as several seconds or minutes, based on the heating temperature, the external temperature, and the current transmission period of the second trigger signal. can

At this time, if the expected rising temperature after a predetermined time exceeds the first threshold value, or if the average slope value of the expected rising temperature curve after a predetermined time from the present is greater than or equal to a certain value, the current heating temperature of the infrared LED is less than or equal to the first threshold even if, the trigger condition monitoring unit 130 may be a second pre-fold increase n ₁ the transmission period of the trigger signal.

However, even in this case, the trigger condition monitoring unit 130 may pre-increase the _{transmission period of the second trigger signal by n 1 times only under certain conditions.}

For example, since increasing the transmission period of the second trigger signal may inaccurately monitor the driver's condition, it may be increased only when the driving state of the driver's vehicle does not have an accident risk.

For example, as in an embodiment to be described later, when the light intensity is less than or equal to the third threshold and/or the traffic conditions around the vehicle analyzed through the black box image (eg, accident risk factors, traffic volume, etc.) Only when the score is less than or equal to the fifth threshold and/or when the converted score of navigation information (eg, the type of the current driving road, the current driving speed, the amount of traffic on the current driving road, etc.) is less than or equal to the seventh threshold, Although the present heating temperature of the infrared LED below a first threshold value, a trigger condition monitoring unit 130 may be a second pre-fold increase n the _first transmission period of the trigger signal.

second embodiment

Referring to FIG. 4 , the trigger condition monitoring unit 130 acquires luminance information obtained by measuring the luminance of a light source around the driver based on the captured image of the driver received from the image analysis unit 150 through the camera unit 200 . It can be done (S405).

For example, the image analyzer 150 may acquire luminance information of a light source around the driver based on the captured image, and transmit it to the trigger condition monitoring unit 130 .

The trigger condition monitoring unit 130 determines whether the light intensity of the light source exceeds a second threshold value based on the light intensity information (S410), and when the light intensity exceeds the second threshold value, the transmission period of the second trigger signal is 1/ It can be reduced to _{n 2 .} That is, when the luminous intensity is greater than the second threshold value, the transmission period is set to be short, the time during which the operation of the infrared LED is stopped can be set to be short, and the infrared LED can be controlled to operate as often as necessary.

However, when the luminance is less than or equal to the second threshold and exceeds the third threshold, the current transmission period of the second trigger signal may be maintained. If the luminous intensity is less than or equal to the third threshold, the transmission period of the second trigger signal may be increased _{to n 2 .} That is, if the luminous intensity is less than or equal to the second threshold, the transmission period may be lengthened to increase the time period during which the operation of the infrared LED is stopped, thereby reducing the heating phenomenon of the infrared LED (S415).

In this case, n ₂ may be a positive number.

On the other hand, if the light source around the driver exceeds the second threshold value for more than a predetermined time, the second trigger signal is continuously transmitted in a short period, so that the heating temperature of the infrared LED may increase.

Therefore, in this case, even if the light source around the driver exceeds the second threshold value, it should be possible to reduce the heating temperature of the infrared LED by increasing the transmission period of the second trigger signal.

Accordingly, even if the luminous intensity exceeds the second threshold value, in a specific case, the transmission period of the second trigger signal may be maintained or _{increased to n 2 without decreasing to 1/n 2 .} For example, as in one embodiment to be described later, when the converted score of the traffic situation (eg, accident risk factor, traffic volume, etc.) around the vehicle analyzed through the black box image is less than or equal to the fifth threshold and/ Alternatively, if the converted score of navigation information (for example, the type of the current driving road, the current driving speed, the amount of traffic on the current driving road, etc.) is less than or equal to the seventh threshold, the need to closely monitor the driver's condition is reduced. , even if the luminous intensity exceeds the second threshold value, the transmission period of the second trigger signal may be maintained or _{increased to n 2 without decreasing to 1/n 2 .}

third embodiment

Referring to FIG. 5 , the trigger condition monitoring unit 130 may acquire a black box image installed in the vehicle through the communication unit 700 ( S505 ).

Information on traffic conditions around the vehicle may be analyzed and obtained through the black box image (S510).

For example, it is possible to analyze accident risk factors and traffic volume around the vehicle in the black box image.

The trigger condition monitoring unit 130 may determine the transmission period of the second trigger signal based on the information on the traffic situation (S515).

For example, if the sum of the first score converted from the accident risk factors around the vehicle and the second score converted based on the surrounding traffic exceeds the fourth threshold, the transmission period of the second trigger signal is 1/n ₃ can be reduced to If the sum of the first score and the second score is less than or equal to the fourth threshold and exceeds the fifth threshold, the transmission period of the second trigger signal may be maintained without changing. On the other hand, when the sum of the first score and the second score is equal to or less than the fifth threshold, the second trigger signal may increase the _{transmission period to n 3 .} In this case, n ₃ may be a positive number.

delete

4th embodiment

Referring back to FIG. 5 , the trigger condition monitoring unit 130 obtains navigation information of the road on which the vehicle is currently traveling from a navigation system or a navigation device or a navigation server linked to the vehicle through the communication unit 700 . It can be (S505 to S510).

In addition, the trigger condition monitoring unit 130 may determine the transmission period of the second trigger signal based on the navigation information (S515).

For example, the third score according to the type of road on which the vehicle is traveling, the fourth score according to the current driving speed of the vehicle, and the fifth score according to the amount of traffic on the road are added, and the summed value is the sixth threshold. If the value is exceeded, the transmission period of the second trigger signal may be reduced _{to 1/n 4 .} If the sum of the third score, the fourth score, and the fifth score is less than or equal to the sixth threshold and exceeds the seventh threshold, the transmission period of the second trigger signal may be maintained without changing. On the other hand, when the sum of the third score, the fourth score, and the fifth score is equal to or less than the seventh threshold, the second trigger signal may increase the _{transmission period to n 4 .} In this case, n ₄ may be a positive number.

For example, as for the third score, when the road on which the vehicle travels is a highway, there may be fewer unexpected situations, so a lower score may be given than when the road on which the vehicle travels is a city center city. In other words, the third score is given a lower score on a motor-only road such as a highway than on a national road, a local road, and a city/county/gu in which a crosswalk is installed.

For example, the fourth score may be given a higher score as the driving speed of the vehicle increases. In this case, when the driving speed of the vehicle is higher than the regular speed of the road on which the vehicle is currently driven, the interval between the points given may be increased. For example, if driving at 40 km/h on a road where the speed limit is 60 km/h, the fourth score is 40, and if driving at 50 km/h is given 50 points, when driving at 100 km/h, 20% By adding , 120 points may be awarded.

For example, the fifth score may be awarded according to the amount of traffic at the current location of the vehicle. For example, the amount of traffic at the location may be classified into 'promotion', 'slow run', 'congestion', etc., and scores may be given in the order of 'want', 'slow run', and 'congestion' .

Meanwhile, the above-described first to fourth embodiments may be implemented independently, but may be implemented in combination. For example, the trigger condition monitoring unit 130 may determine the transmission period of the second trigger signal by comprehensively considering temperature information, image brightness, black box image information, and navigation information.

For example, the factor for the transmission period determined according to the first embodiment is 1/n ₁ , the factor for the transmission period determined according to the second embodiment is 1 (ie, maintain the current period), and the second 3 If the factor for the transmission period determined according to the embodiment is n ₃ and the factor for the transmission period determined according to the fourth embodiment is 1/n ₄ , the trigger condition monitoring unit 130 is the second trigger signal. A period that is (1/n ₁ * 1 * n ₃ * 1/n ₄ ) times the current transmission period may be determined as the transmission period of the second trigger signal, and accordingly, the second trigger signal may be transmitted to operate the infrared LED.

In addition, the present invention may further include the following features.

For example, the controller 100 may transmit an initial boot signal for booting the plurality of cameras to the camera unit 200 and/or the plurality of cameras.

Here, the initial booting signal refers to a command for simultaneously starting the operation of each of the plurality of cameras, a command for sequentially starting the operation of each of the plurality of cameras, and turning on the power of each of the plurality of cameras. ) may include a command for

Meanwhile, the controller 100 may set and/or reset a timing and/or a period for transmitting an initial booting signal to the camera unit 200 and/or a plurality of cameras. Alternatively, the timing and/or period at which the communication unit 700 transmits the initial booting signal to the camera unit 200 and/or the plurality of cameras may be set and/or reset by the control unit 100 .

The communication unit 700 and/or the control unit 100 may receive a sync signal (and/or a response signal) generated based on the initial booting signal from the plurality of cameras.

For example, a sync signal and/or a response signal may be used to synchronize the plurality of cameras and an electronic control unit (ECU) of a vehicle terminal. For example, the sync signal may be used for synchronization of a reference camera among the plurality of cameras, the remaining cameras among the plurality of cameras, and the ECU.

In addition, the sync signal and/or the response signal may be generated by each of the plurality of cameras and/or by the controller 100 based on the initial booting signal.

For example, each of the plurality of cameras and/or the control unit 100 may include a first timing at which the initial booting signal is transmitted by the communication unit 700 of the vehicle terminal and the camera unit. A sync signal and/or a response signal may be generated based on a difference between second timings received by 200 .

Based on the sync signal and/or the response signal, the controller 100 may calculate and/or check the booting time differences between the 'reference camera' and 'each of the remaining cameras' excluding the reference camera among the plurality of cameras.

For example, when there is one reference camera and the number of the remaining cameras (No. 1 camera, No. 2 camera, ..., No. N camera) is N, the vehicle terminal determines the reference based on the sync signal. A first booting time difference between the camera and the first camera, a second booting time difference between the reference camera and the second camera, ... , and an Nth booting time difference between the reference camera and the Nth camera may be obtained.

The controller 100 may generate a boot sequence (and/or a sequence for boot time difference compensation) for compensating for the booting time differences based on the booting time differences.

The controller 100 may change the initial booting signal (ie, the first booting signal) into an updated booting signal (ie, the second booting signal) based on the booting sequence (and/or the booting time difference compensation sequence). . In other words, the second booting signal may be a reset of the first booting signal based on the booting sequence (and/or the booting time difference compensation sequence).

For example, by applying the booting sequence (and/or booting time difference compensation sequence) to the initial booting signal based on a predefined rule, the updated booting signal (ie, the second booting signal) can be obtained. have.

The controller 100 may transmit the updated booting signal to the plurality of cameras.

For example, a camera physically closest to the controller 100 among the plurality of cameras may have the shortest initial booting time. The initial booting time may be defined and/or set as a time at which the camera closest to the controller 100 is booted by the initial booting signal.

As another example, the initial booting time of a camera other than the one physically closest to the controller 100 among the plurality of cameras may be the shortest. In this case, the initial booting time may be defined and/or set as a time at which the other camera is booted by the initial booting signal.

As another example, even if the initial booting time of the camera physically closest to the controller 100 among the plurality of cameras is the shortest, if a predetermined criterion is satisfied (or does not satisfy the predetermined criterion), The initial booting time may be defined and/or set as a time at which a camera other than the closest camera to the controller 100 is booted by the initial booting signal.

Here, the predetermined criterion means that when a user (driver) of a vehicle (vehicle) starts the engine, he/she feels a sense of heterogeneity related to camera booting (eg, Around View Monitor (AVM), Surround View Screen). (SVM, Surround View Monitor) may be satisfied when a boot environment check message is input through the input unit 500 (in the case that a part of the screen is not visible, that is, when a part of the screen is not output). have.

In addition, the control unit 100 may generate and/or monitor information about an Around View Monitor (AVM) and/or a Surround View Monitor (SVM) installed in a vehicle (vehicle), and the Set a plurality of grid pattern areas and/or a plurality of viewing areas in the around view screen and/or the surround view screen, and blind (and/or blackout) in the plurality of grid areas and/or the plurality of viewing areas. ) to be identified at least one region (and/or region). the number of at least one zone (and/or zone) that are blinded (and/or blacked out) within the plurality of grid areas and/or the plurality of viewing zones is greater than or equal to a first threshold, and/or the plurality of If the ratio of the number of at least one area (and/or area) to be blinded (and/or blacked out) to the number of grid-patterned areas (and/or plurality of viewing areas) is equal to or greater than the second threshold, the predetermined criterion is can be considered satisfactory.

For example, the camera unit 200 and/or the plurality of cameras are two cameras located in front of the vehicle terminal, two cameras located in the front side of the vehicle terminal, and side and rear of the vehicle terminal. It may include two cameras positioned at the rear and one camera positioned at the rear of the vehicle terminal.

For example, the reference camera may be one camera positioned at the rear of the vehicle terminal or one camera among two cameras positioned in front of the vehicle terminal.

For example, the reference camera may be a camera having the shortest initial booting time among the plurality of cameras. The initial booting time may indicate a time at which the plurality of cameras are booted by the initial booting signal.

For example, as the reference camera, at least one of a plurality of cameras of the camera unit 200 may be determined and/or set by the controller 100 .

For example, the camera unit 200 and/or each of the plurality of cameras may periodically transmit 30 or 60 images per second to the controller 100 .

As described above, while this specification contains numerous specific implementation details, these are not to be construed as limiting as to the scope of any invention or claim, but rather may be specific to particular embodiments of a particular invention. It should be understood as a description of features. Certain features that are described herein in the context of separate embodiments may be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments, either individually or in any suitable subcombination. Furthermore, although features operate in a particular combination and may be initially depicted as claimed as such, one or more features from a claimed combination may in some cases be excluded from the combination, the claimed combination being a sub-combination. or a variant of a sub-combination.

Likewise, although acts are depicted in the figures in a particular order, it should not be construed that all acts shown must be performed or that such acts must be performed in the specific order or sequential order shown in order to achieve desirable results. In certain cases, multitasking and parallel processing may be advantageous. Further, the separation of the various system components of the above-described embodiments should not be construed as requiring such separation in all embodiments, and the program components and systems described may generally be integrated together into a single software product or packaged into multiple software products. You have to understand that you can.

Certain embodiments of the subject matter described herein have been described. Other embodiments are within the scope of the following claims. For example, acts recited in the claims may be performed in a different order and still achieve desirable results. As an example, the processes illustrated in the accompanying drawings do not necessarily require the specific illustrated order or sequential order to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous.

The present description sets forth the best mode of the invention, and provides examples to illustrate the invention, and to enable a person skilled in the art to make and use the invention. This written specification does not limit the present invention to the specific terms presented. Accordingly, although the present invention has been described in detail with reference to the above-described examples, those skilled in the art can make modifications, changes and modifications to the examples without departing from the scope of the present invention.

Therefore, the scope of the present invention should not be defined by the described embodiment, but should be defined by the claims.

The present invention relates to a method of operating an infrared LED (Lighting Emitting Diode) for monitoring a driver's movement and a driver monitoring system including the infrared LED.

According to the present invention, the camera speed and the infrared LED can be controlled according to the temperature of the infrared LED and the monitoring algorithm, so that the temperature management of the infrared LED is easy and the performance of the monitoring algorithm can be improved.

Therefore, the present invention can contribute to the overall development of the autonomous driving industry through a method of operating an infrared LED (Lighting Emitting Diode) for monitoring a driver's movement and a driver monitoring system including the infrared LED, and can be marketed or sold There is industrial applicability because the possibility of

100: control unit 200: camera unit
300: infrared LED unit 400: storage unit
500: input unit 600: output unit
700: communication department

Claims (5)

In the method for monitoring a driver by controlling the operation of the infrared LED by the camera control unit,
When power is input to the camera control unit through the power unit, a booting signal is transmitted to the camera,
Transmitting a first trigger signal for operating the camera to the camera, and transmitting a second trigger signal for operating the infrared LED to the infrared LED based on a predetermined transmission period, - The infrared LED operates for a predetermined time after receiving the second trigger signal, but does not operate if the second trigger signal is not further received even after the predetermined time has elapsed -
Receive an image of the driver from the camera,
Analyzing the captured image, comprising monitoring the driver's condition,
Obtaining luminous intensity information by measuring the luminous intensity of the driver's ambient light source based on the analysis result of the photographing,
Measuring the internal heating temperature of the infrared LED through a first temperature sensor installed inside the infrared LED,
Measuring the external measurement temperature of the infrared LED through a second temperature sensor installed on the outside of the infrared LED,
Extracting the expected rising temperature curve of the infrared LED in consideration of the internal heating temperature, the external measurement temperature and the transmission period,
When the internal heating temperature of the infrared LED exceeds the first threshold value, the transmission period is reset to a longer value, and when the internal heating temperature of the infrared LED is less than the first threshold value, the transmission period is maintained. ,
When the slope of the expected rising temperature curve exceeds a second threshold value, control to increase the transmission period n times even if the internal heating temperature of the infrared LED is less than the first threshold value, wherein n is a positive value ,
How to monitor drivers.
delete delete delete The method of claim 1,
Based on the driver's state, further comprising outputting a warning sound,
How to monitor drivers.

Images