KR20190001029A - Passenger cognition apparatus in vehicle and method thereof - Google Patents

Abstract

The present invention relates to a passenger cognition apparatus in a vehicle and a method thereof, wherein the passenger cognition apparatus comprises: an infrared ray (IR) image input unit receiving an IR image photographed with at least one IR camera mounted in a vehicle; a learning DB storing learning data to recognize a passenger in the vehicle; and a control unit recognizing the passenger in the vehicle by analyzing the IR image inputted through the IR image input unit based on the learning data stored in the learning DB.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a passenger identification system,

The present invention relates to an apparatus and method for recognizing a passenger of a vehicle, and more particularly, to a passenger identification system and method for recognizing a passenger of a passenger by recognizing a passenger of the vehicle and providing passenger information to an electronic control unit (ECU) Belt ' s occupant recognition apparatus and method for warning the wearer or controlling the airbag and the like.

An airbag system mounted on a vehicle is an apparatus for minimizing injury by instantaneously inflating an airbag between a driver and a steering wheel or between a passenger in a passenger seat and an instrument panel in the event of a collision.

In such an airbag drive apparatus, once the airbag is driven by the impact of the vehicle, the airbag is operated at a constant speed and pressure regardless of the adult or child. Therefore, a very serious shock will be applied to passengers in a flash, so if the airbag is operated while the child is on board, a fatal accident may occur to the child. Therefore, it is necessary to detect whether the occupant of each seat is an adult or a child and to recognize when a child is aboard, to reduce the size of the airbag and to slow down the airbag emergence speed.

On the other hand, when a passenger wears a seatbelt or when he / she is not wearing a seatbelt, the impact received by the airbag is different. That is, when the passenger wears the seatbelt, even if the vehicle is impacted by a collision, the inertia causes less forward deviation. Therefore, when the passenger (or passenger) senses whether or not the seat belt has been worn, it is necessary to output a warning if the seat belt is not worn, or to reduce the size of the air bag and decelerate the air bag.

In order to control the functions of the in-vehicle electronic devices (e.g., airbags, seat belts, etc.) in accordance with the passengers, it is necessary to recognize the passengers in the vehicle more accurately and to provide the passenger information to the function- .

The background art of the present invention is disclosed in Korean Patent Laid-Open Publication No. 10-1998-0001491 (published on Mar. 30, 1998, an air bag control apparatus using a charge coupled device and a control method thereof).

SUMMARY OF THE INVENTION According to an aspect of the present invention, there is provided an electronic control unit (ECU) in a vehicle which recognizes a passenger of a vehicle and provides passenger information to the passenger, And it is an object of the present invention to provide a vehicle occupant recognition apparatus and method for warning a wear of a seat belt or controlling an airbag and the like.

A passenger recognizing apparatus for a vehicle according to an aspect of the present invention includes an IR image input unit for receiving an IR image photographed using at least one IR (infrared ray) camera mounted in a vehicle; A learning DB for storing learning data for recognition of passengers in the vehicle; And a control unit for analyzing the IR image input through the IR image input unit based on the learning data stored in the learning DB to recognize a passenger in the vehicle.

The present invention further includes a sensor unit that detects a state of the vehicle interior and exterior using at least one sensor installed in the vehicle, wherein the sensor unit detects at least one of a seat belt sensor, a seat sensor, an illuminance sensor, And a control unit.

In the present invention, the controller updates learning data stored in the learning DB through a learning server or a learning server connected through a wired / wireless network.

In the present invention, the control unit may transmit the IR image to a learning server or a learning server connected through a wired / wireless network to provide learning information for recognizing a passenger, and the learning server or the learning server may use the received IR image And updates the learning data.

In the present invention, the learning DB includes an offline-type database directly connected to a circuit or a cloud-based database connected online.

The present invention further includes a communication unit for communicating with a learning server or a learning server to receive learning data for recognizing a passenger, wherein the control unit transmits the IR image to the learning server or the learning server through the communication unit .

In the present invention, the control unit transmits the recognized passenger information to the function-specific electronic control unit (ECU) provided in the vehicle, and the electronic control unit (ECU) controls the cluster or the infotainment To output a seat belt warning through the airbag, or to control the deployment of the airbag for a passenger in the event of a vehicle collision.

In the present invention, the learning DB is learned by reflecting the vehicle internal environment information and the vehicle external environment information, is learned by reflecting the object information and the car seat standard information, and distinguishes between adults and children who do not use a car seat Learning is performed by reflecting the reference information, reflecting the appearance of wearing or not wearing the object, and storing the learned learning data reflecting the pose or action.

In the present invention, the learning data is data learned so that a passenger can be recognized using an IR image synthesized in an AND logic combination form of the learned information.

In the present invention, the control unit analyzes the IR image based on the learning data stored in the learning DB to recognize non-passengers, adult and child passengers, and infant passengers.

In the present invention, the controller may be implemented by applying a convolutional neural network and an artificial neural network.

According to another aspect of the present invention, there is provided a method of recognizing a passenger in a vehicle, the method comprising the steps of: when the vehicle is started, capturing an IR image of a vehicle using at least one IR camera; Analyzing the IR image based on the learning data stored in the learning DB to recognize the passenger; And the control unit sends the recognized passenger information to the function-specific electronic control unit (ECU) in the vehicle so that the electronic control unit (ECU) outputs a seat belt warning through the cluster or infotainment (IVI) device according to its own function Or controlling the deployment of the airbag in accordance with the passenger in the event of a vehicle collision.

In the present invention, after the step of photographing the IR image, the control unit transmits the IR image to a learning server or a learning server, and the learning server or the learning server continues the existing learning data through learning of the transmitted IR image The method comprising the steps of:

According to another aspect of the present invention, there is provided a method of updating a learning data of a learning DB, the method comprising the steps of: receiving the latest learning data from the learning server or the learning server;

In the present invention, the learning DB is learned by reflecting the vehicle internal environment information and the vehicle external environment information, is learned by reflecting the object information and the car seat standard information, and distinguishes between adults and children who do not use a car seat Learning is performed by reflecting the reference information, reflecting the appearance of wearing or not wearing the object, and storing the learned learning data reflecting the pose or action.

In the present invention, the learning data is data learned so that a passenger can be recognized using an IR image synthesized in an AND logic combination form of the learned information.

In the present invention, in the step of recognizing the passenger, the control unit analyzes the IR image based on the learning data stored in the learning DB to recognize non-passengers, adult and child passengers, and infant passengers .

According to one aspect of the present invention, the present invention recognizes a passenger of a vehicle and provides passenger information to a function-based electronic control unit (ECU) in the vehicle, thereby alerting the passenger of wearing a seat belt, And so on.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram showing an outline configuration of a passenger recognizing apparatus for a vehicle according to an embodiment of the present invention; FIG.
FIG. 2 is an exemplary diagram showing a configuration for explaining a passenger recognition operation of the control unit in FIG. 1; FIG.
3 is a flowchart illustrating a method of recognizing a passenger of a vehicle according to an embodiment of the present invention.
FIG. 4 is an exemplary diagram for explaining vehicle interior environment information reflected in learning data in FIG. 1; FIG.
FIG. 5 is an exemplary diagram for explaining vehicle exterior environment information reflected in the learning data in FIG. 1; FIG.
FIG. 6 is an exemplary diagram for explaining the object information and the car seat standard information reflected in the learning data in FIG. 1; FIG.
FIG. 7 is an exemplary diagram for explaining reference information for distinguishing adults and children reflected in learning data in FIG. 1; FIG.
FIG. 8 is an exemplary diagram for explaining various appearance information reflected in the learning data in FIG. 1; FIG.
FIG. 9 is an exemplary diagram for explaining various poses and behaviors reflected in the learning data in FIG. 1; FIG.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of a vehicle occupant recognition apparatus and method according to the present invention will be described with reference to the accompanying drawings.

In this process, the thicknesses of the lines and the sizes of the components shown in the drawings may be exaggerated for clarity and convenience of explanation. In addition, the terms described below are defined in consideration of the functions of the present invention, which may vary depending on the intention or custom of the user, the operator. Therefore, definitions of these terms should be made based on the contents throughout this specification.

FIG. 1 is an exemplary diagram showing a schematic configuration of a passenger recognizing apparatus for a vehicle according to an embodiment of the present invention.

1, the passenger recognizing apparatus for a vehicle according to the present embodiment includes an IR (infrared ray) image input unit 110, a sensor unit 120, a control unit 130, a learning DB 140, (150).

The IR image input unit 110 receives IR images photographed using at least one IR camera mounted in the vehicle.

The IR camera is installed in front of the vehicle or on the ceiling.

For example, the IR camera may be installed in any one of a vehicle dashboard, a room mirror, and a ceiling above each seat row (e.g., one row, two rows, three columns, etc.) Respectively. In addition, unlike a general visible light camera (e.g., CCD, CMOS), the IR camera captures an image by sensing heat. For example, the IR camera is characterized in that a hot spot is photographed in red and a cold spot is photographed in green.

The reason for using the IR camera in this embodiment is to recognize the passenger at night, and in general, a visible light camera (e.g., CCD, CMOS) is too sensitive to sunlight, so an IR camera is used to improve stability. However, an infrared filter may be attached to a visible light camera (eg CCD, CMOS).

The sensor unit 120 detects the inside and outside of the vehicle using at least one sensor installed in the vehicle.

For example, the sensor unit 120 includes a seat belt sensor, a seat sensor, an illuminance sensor, a temperature sensor, and the like, and may additionally include other sensors not described in this embodiment for convenience.

The control unit 130 analyzes the IR image input through the IR image input unit 110 based on the learning data stored in the learning DB 140, and recognizes the passenger.

The learning data stored in the learning DB 140 may be updated through a server (e.g., a running server, a learning server, etc.) (not shown) connected through a wired / wireless network.

The control unit 130 may receive learning data from a server (e.g., a running server, a learning server, etc.) (not shown) connected through the wired / wireless network and update the learning DB 140.

The control unit 130 transmits the IR image input through the IR image input unit 110 to the server (e.g., a running server, a learning server, etc.) (not shown) connected through the communication unit 150, (I. E. Learning material for passenger recognition). Of course, the IR image transmitted to the server (not shown) may be information included in the range already already learned by the server (not shown), but regardless of the IR image, the server (not shown) So that the updated learning data can be generated.

The learning DB 140 stores learning data for recognizing the passenger.

The learning DB 140 may be a database in an offline manner (i.e., a scheme directly connected to a circuit) or a database in an online scheme (e.g., a wireless cloud scheme).

The communication unit 150 communicates with the server (e.g., a running server, a learning server, etc.) (not shown) connected in a wireless (or wired) manner.

The communication unit 150 transmits and receives at least one piece of information (e.g., learning data, IR image, etc.) to the server (e.g., a running server, a learning server, etc.) (not shown) using any one of the communication methods described above.

The wireless scheme includes a wireless communication scheme using at least one of WiFi, Bluetooth, NFC (Near Field Communication), mobile communication (3G, 4G, 5G, etc.), and infrared rays. The wired method includes a wired communication method using at least one of a USB (Universal Serial Bus), a LAN (Local Area Network), a serial line, and a parallel line. However, the wireless / wired communication method described above is only described by way of example and is not described for limiting the communication method.

Meanwhile, the control unit 130 analyzes the IR image and transmits the recognized passenger information to the function-specific electronic control unit (ECU) 160 installed in the vehicle.

Accordingly, the electronic control unit (ECU) 160 can output a warning (e.g., a seat belt warning) through a cluster or an infotainment (IVI) device according to its own function, (Projecting) of the airbag according to a user's request (for example, adult, child, infant, etc.).

2 is a diagram illustrating an example of a configuration for explaining a passenger recognition operation of the controller 130 in FIG.

2, the control unit 130 includes a non-passenger recognition unit 131, an adult and child passenger recognition unit 132, and an infant passenger recognition unit 133. [

The controller 130 receives vehicle interior environment information and vehicle exterior environment information.

FIG. 4 is an exemplary view for explaining the vehicle internal environment information, and FIG. 5 is an exemplary view for explaining the vehicle external environment information.

As shown in Fig. 4, when it is assumed that a passenger is boarded on a vehicle having a two-row seat, ①, ②, ③, ④ and ⑤ show the position of the seat (seat) Ⓑ indicates the distance that the backrest can move forward and backward when looking at the seat (seat) from the side, and ⓒ indicates the distance that the seat can be moved up and down when viewed from the side. Represents the distance.

Accordingly, the seat setting a in Fig. 4 (a) indicates that the distance by which the seat (seat) can be moved forward and backward is set by quadrupling, and the seat setting b is set by quadrupling the distance that the backrest can be moved back and forth , And the seat setting C indicates that the distance that the seat (seat) can be moved up and down is set by quadrant.

4 (b) and 4 (c), the seat (seat) setting is set to a minimum distance (0%) in which the seat (seat) and the backrest can be moved back and forth or up and down, (100%), and can be set considering various persons' keys.

Also, as shown in FIG. 5, the illuminance state according to the weather and time (for example, morning, noon, evening, and night), the illuminance state in which the light reflected to the passenger at random is considered in the surroundings, (For example, when the vehicle door is opened or closed, the vehicle door is opened or closed, the vehicle is shaken during traveling, or the vehicle is moving due to movement of the passenger) continuously within 2 seconds, Or more) may be reflected.

Accordingly, the learning data stored in the learning DB 140 naturally reflects various types of vehicle interior environment information as shown in FIG. 4, and data learned by reflecting the various vehicle exterior environment information as shown in FIG. to be.

Also, the control unit 130 (i.e., the non-occupant recognition unit 131) receives object information and car seat information for recognizing non-passengers (e.g., objects, animals, car seats, etc.).

Naturally, the learning data stored in the learning DB 140 is data learned by reflecting various article information and car seat standard information as shown in FIG.

Here, the item information means, for example, a box, a bag, an animal, a food, and the like excluding the passenger (person). However, the car seat information for the infant on the above object information is separately reflected in the learning separately from the other object information.

On the other hand, the car seat standard shown in Fig. 6 is exemplarily described for the sake of explanation.

In addition, the control unit 130 (i.e., the adult and child recognition unit 132) may be classified as a passenger (for example, an adult, a passenger who does not use a car seat as a child, (Type), appearance (appearance), posture & behavior information are input for recognition.

In addition, the control unit 130 (that is, the infant occupant recognizing unit 133) inputs car seat information, appearance (external shape), vehicle interior environment information, and vehicle exterior environment information for recognizing a passenger (for example, a passenger using a car seat) Receive.

In order for the controller 130 to recognize the passenger through the analysis of the IR image, it is needless to say that the learning data stored in the learning DB 140 is various vehicle internal environment information as shown in FIG. 4, And is learned by reflecting various vehicle external environment information as shown in FIG. 6, reflected by reflecting various object information and car seat standard information as shown in FIG. 6, and is classified into adult (male, female) and child 8 (a) and 8 (b), and various kinds of appearance (for example, wearing a hat, using a face, etc.) Etc.), and are learned by reflecting various poses or behaviors (for example, left-right rotation, left-right tilting, bowing or crouching, steering wheel turning operation, etc.) as shown in FIG.

At this time, the learning data does not limit the posture of the limbs (that is, the limb posture is not a factor that greatly influences the identification of the passengers).

On the other hand, the learning data is used to recognize the passenger (for example, non-passenger, adult, child, infant, etc.) using the IR image synthesized in the AND logic combination form described in FIGS. 4 to 9 This means the data learned to be used. It should be noted that the condition (reference) shown in FIG. 4 to FIG. 9 is exemplarily described for the sake of explanation, so that it is possible to apply more various conditions (reference) during actual learning.

As described above, the controller 130 analyzes the IR image input through the IR image input unit 110 based on the learning data stored in the learning DB 140, and recognizes the passenger. At this time, the controller 130 may apply a convolutional neural network (CNN) and an artificial neural network (ANN) to improve the performance.

3 is a flowchart illustrating a method of recognizing a passenger of a vehicle according to an embodiment of the present invention.

As shown in FIG. 3, when the vehicle is started (S101), the control unit 130 takes an IR image of the inside of the vehicle using at least one IR camera (S102).

When the IR image of the vehicle is captured as described above, the controller 130 analyzes the IR image based on the learning data stored in the learning DB 140 (S103) and recognizes the passenger (S104).

Since the learning data is data learned to recognize the passenger using the IR image synthesized in the AND logic combination form described with reference to FIGS. 4 to 9, The passenger can be recognized through the analysis of the IR image.

When the passenger of the vehicle is recognized as described above, the controller 130 transmits the recognized passenger information to the electronic control unit (ECU) 160 in the vehicle (S105), and the electronic control unit (ECU) 160 ) Can output alerts (such as seat belt warnings) via clusters or infotainment devices (IVI), depending on their capabilities, and can be used to alert passengers (eg, adults, children, infants, etc.) It is possible to control the deployment (projection) of the airbag accordingly.

The control unit 130 may further transmit the IR image to a learning server (not shown) (S106) so that the learning server (not shown) continuously updates the existing learning data through the learning of the transmitted IR image .

If there is an update issue periodically, the controller 130 receives the latest learning data from the learning server (not shown) and updates the learning data of the learning DB 140 (S107).

As described above, the present embodiment more accurately recognizes the passengers of the vehicle and provides the occupant information to the function-specific electronic control unit (ECU) in the vehicle, thereby alerting the passenger of the wearing of the seat belt, So that the safety of the vehicle can be improved.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, I will understand the point. Accordingly, the technical scope of the present invention should be defined by the following claims.

110: IR image input unit 120:
130: control unit 131: non-passenger recognition unit
132: Adult and Child Passenger Recognition Unit 133: Infant Passenger Recognition Unit
140: learning DB 150: communication section
160: Electronic control unit

Claims (17)

An IR image input unit for receiving an IR image photographed using at least one IR (infrared ray) camera mounted inside the vehicle;
A learning DB for storing learning data for recognition of passengers in the vehicle; And
And a control unit for analyzing the IR image input through the IR image input unit based on the learning data stored in the learning DB to recognize a passenger in the vehicle.
The method according to claim 1,
And a sensor unit for detecting a state of the inside and the outside of the vehicle using at least one sensor installed in the vehicle,
Wherein the sensor unit includes at least one of a seat belt sensor, a seat sensor, an illuminance sensor, and a temperature sensor.
The apparatus of claim 1,
The learning data stored in the learning DB is updated through a learning server or a learning server connected through a wired / wireless network.
The apparatus of claim 1,
The IR image is transmitted to a learning server or a learning server connected through a wired / wireless network to provide learning data for recognizing a passenger,
Wherein the learning server or the learning server learns and updates the learning data using the transmitted IR image.
The learning DB according to claim 1,
An offline database that is directly connected to the circuit,
And a database of a cloud system connected on-line.
The method according to claim 1,
And a communication unit for communicating with a learning server or a learning server to receive learning data for recognizing a passenger,
Wherein the control unit transmits the IR image to the learning server or the learning server through the communication unit.
The apparatus of claim 1,
The control unit transmits the recognized passenger information to the function-specific electronic control unit (ECU) installed in the vehicle so that the electronic control unit (ECU) outputs the safety belt warning via the cluster or infotainment (IVI) Thereby enabling control of airbag deployment for a passenger in the event of a vehicle collision.
The learning DB according to claim 1,
Vehicle interior environment information, and vehicle exterior environment information,
It is learned by reflecting object information and car seat standard information,
It is learned by reflecting standard information for distinguishing children who do not use adult and car seat,
They are taught to reflect the appearance of wearing or not wearing things, and
And the learned learning data is stored by reflecting a pose or an action.
9. The method according to claim 8,
And data learned so that the passenger can be recognized by using the IR image synthesized in the AND logic combination form of the learned information.
The apparatus of claim 1,
Analyzes the IR image based on learning data stored in the learning DB,
Recognizes non-passengers, adult and child passengers, and infant passengers.
The apparatus of claim 1,
A convolutional neural network and an artificial neural network may be applied to the passenger identification device of the vehicle.
When the vehicle is started, photographing an IR image of the inside of the vehicle using the at least one IR camera;
Analyzing the IR image based on the learning data stored in the learning DB to recognize the passenger; And
The controller transmits the recognized passenger information to the function-specific electronic control unit (ECU) in the vehicle so that the electronic control unit (ECU) outputs the seat belt warning through the cluster or infotainment (IVI) device according to its own function And controlling the deployment of the airbag for a passenger in the event of a vehicle collision.
13. The method according to claim 12, wherein after the step of photographing the IR image,
The control unit transmits the IR image to a learning server or a learning server,
And allowing the learning server or the learning server to continuously update the existing learning data through learning of the transmitted IR image.
14. The method of claim 13,
Further comprising the step of the controller receiving the latest learning data from the learning server or the learning server and updating the learning data of the learning DB periodically or in case of an update issue.
13. The learning system according to claim 12,
Vehicle interior environment information, and vehicle exterior environment information,
It is learned by reflecting object information and car seat standard information,
It is learned by reflecting standard information for distinguishing children who do not use adult and car seat,
They are taught to reflect the appearance of wearing or not wearing things, and
And the learned learning data is stored reflecting the pose or behavior.
16. The method according to claim 15,
And means for recognizing the passenger using the IR image synthesized in the AND logic combination form of the learned information.
13. The method of claim 12, wherein in recognizing the passenger,
Wherein the controller analyzes the IR image based on learning data stored in the learning DB to recognize non-passengers, adult and child passengers, and infant passengers.

Images