KR20200082515A - Vehicle information device having vision sensor - Google Patents

Abstract

According to an exemplary embodiment of the present invention, a vehicle information device having a vision sensor includes: a vision sensor provided in a vehicle; and a control part analyzing data obtained by the vision sensor, wherein the control part can determine an object within a detecting range of the vision sensor. Therefore, the vehicle information device having the vision sensor can improve a vehicle situation monitoring efficiency.

Description

VEHICLE INFORMATION DEVICE HAVING VISION SENSOR}

One embodiment of the present invention relates to a vehicle information device having a vision sensor.

Technology for detecting various motions or gestures using a vision sensor has been developed, and research to achieve various purposes using this technology is continuing.

One aspect of the present invention, it is possible to provide a vehicle information device having a vision sensor that can improve the monitoring efficiency of the vehicle situation.

A vehicle information device having a vision sensor according to an exemplary embodiment of the present invention includes: a vision sensor provided in a vehicle; And a controller that analyzes data acquired by the vision sensor, wherein the controller can determine an object within a detection range of the vision sensor.

At this time, the control unit may determine whether the object is a living or inanimate object.

In addition, the controller may determine whether the creature is an adult or a child.

In addition, further comprising an output unit connected to the control unit, the control unit may generate a command signal to output a seat belt warning is not output to the output unit when the object is determined to be a living thing.

In addition, further comprising a camera, the control unit, a first operation unit for analyzing the data acquired by the vision sensor; A second operation unit to analyze the captured image of the camera; And a third operation unit integrating the operation results of the first operation unit and the second operation unit.

In addition, further comprising a main server for controlling at least some of the matters related to the vehicle, the control unit notifies the main server of the determination result, and if the determination result includes information that determines the object as a living thing, the main The server activates the seat belt warning function, and when the determination result includes information determining the object as inanimate, the main server may deactivate the seat belt warning function.

In addition, further comprising a main server for controlling at least a part of the matters related to the vehicle, the control unit notifies the main server of the determination result, and if the determination result includes information that determines the object as a child, the main The server may generate a control signal such that at least one of the door lock and the window lock of the vehicle is activated.

In addition, the vision sensor may include a first vision sensor located on the ceiling of the vehicle.

In addition, the vision sensor may include a second vision sensor positioned on the seat seat of the vehicle.

In addition, a thermal imaging camera for photographing a thermal image of the object may be further included.

In addition, the control unit may determine the state of the object by analyzing a result of heat distribution of the object.

According to one embodiment of the present invention, it provides a useful effect that the monitoring efficiency of the vehicle situation can be improved.

1 is a block diagram schematically illustrating a vehicle information device having a vision sensor according to an embodiment of the present invention,
2 is a view for explaining the placement position in the vehicle of the vision sensor according to an embodiment of the present invention,
3 is a view for explaining the position and detection range of the vision sensor according to an embodiment of the present invention,
4 is a view for explaining the position and detection range of the vision sensor according to another embodiment of the present invention,
5 is a view schematically illustrating a state viewed from a different angle in the embodiment of FIG. 4,
FIG. 6 is a variation of the embodiment illustrated in FIG. 5.

Advantages and features of the present invention, and a method of achieving them will be clarified by referring to embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms. The present embodiments can be provided to make the disclosure of the present invention complete, and to fully disclose the scope of the invention to those skilled in the art. Throughout the specification, the same reference numerals refer to the same components

The terms used herein are for describing the embodiments and are not intended to limit the present invention. In the present specification, the singular form also includes the plural form unless otherwise specified in the phrase. As used herein,'comprise' and/or'comprising' refers to the elements, steps, operations and/or elements mentioned above, the presence of one or more other components, steps, operations and/or elements. Or do not exclude additions.

In addition, the embodiments described herein will be described with reference to cross-sectional views and/or plan views, which are ideal exemplary views of the present invention. In the drawings, detailed sizes, shapes, thicknesses, curvatures, and the like of each component are exaggerated or schematically illustrated for effective description of technical contents, and their shapes may be modified due to tolerances.

Hereinafter, the configuration and effect of the present invention will be described in more detail with reference to the accompanying drawings.

1 is a block diagram schematically illustrating a vehicle information device having a vision sensor according to an embodiment of the present invention, and FIG. 2 is a view for explaining an arrangement position in a vehicle of the vision sensor according to an embodiment of the present invention 3 is a view for explaining the position and detection range of a vision sensor according to an embodiment of the present invention, and FIG. 4 is a view for explaining the position and detection range of a vision sensor according to another embodiment of the present invention , And FIG. 5 is a diagram schematically illustrating a view of the embodiment of FIG. 4 when viewed from different angles, and FIG. 6 is a modification of the embodiment illustrated in FIG. 5.

Referring to the drawings, a vehicle information device according to an embodiment of the present invention may determine an object including a vision sensor and a control unit.

Here, judging an object may mean judging whether the object is an object, a person, or an animal.

Furthermore, it may mean that it is determined whether an operation performed by the corresponding object corresponds to a predetermined predetermined operation.

In one embodiment, the vision sensor may be a device that performs a function of detecting at least a part of an object where a motion has occurred and generating at least one event signal.

In one embodiment, the vision sensor may include a plurality of sensing elements, and each sensing element may detect the occurrence of a predetermined event and output an event signal.

In one embodiment, the vision sensor may include at least one of an event-based vision sensor and a frame-based vision sensor. Here, the event-based vision sensor may generate an event signal asynchronously in time in response to an event in which light received from an object changes. In one embodiment, the event-based vision sensor may include a dynamic vision sensor (DVS). The dynamic vision sensor responds only to changes in light, and the dynamic vision sensor can detect the outline of a moving object. Accordingly, even if a relatively small amount of data is processed, a predetermined purpose can be realized, so that a target can be detected relatively quickly even when relatively few resources are used.

Meanwhile, the outline of the stationary object may be grasped by moving or vibrating the dynamic vision sensor itself in a predetermined method.

In one embodiment, the frame-based vision sensor may be a sensor that photographs an object in a time-synchronous manner according to a predetermined number of frames per second, and may include a CMOS image sensor (CIS).

Since the above-described event-based vision sensor may utilize only the address of the sensing element or the address and time information of the sensing element that senses the event that the light has changed, the computational amount can be reduced and the power consumption can be reduced. It may have an advantage over a vision sensor.

In one embodiment, the vision sensor may obtain information of an object located within a detection range (eg, field of view (FOV)) and provide it to the control unit. In addition, the controller may determine whether the object is an object, an adult, or a child by analyzing data provided from the vision sensor.

In one embodiment, the detection range of the vision sensor may include each seat disposed inside the vehicle.

In one embodiment, a plurality of seats may be included in the sensing range of one vision sensor, for example, as illustrated in FIG. 5, the third seat (SEAT3) in the sensing range of the sixth vision sensor 110-6 And a fourth sheet SEAT4.

In other embodiments, vision sensors may be provided that are individually matched for each of the seats. For example, as illustrated in FIG. 6, the third sheet SEAT3 is included in the detection range of the 8-1 vision sensor 110-8-1 and the 8-2 vision sensor 110-8-2. A fourth sheet SEAT4 may be included in the sensing range of.

In one embodiment, the control unit may analyze the information provided from the vision sensor to determine that the object located in the specific seat is an object. At this time, in order to determine whether the target object is an object, matters such as whether the object contour is deformed or whether the object contour is moving may be determined. In one embodiment, when it is determined that the time-asynchronous deformation or movement of the contour and the contour of the target object is not a living thing, the target object may be determined to be an object. However, the target object may be a toy doll that can move on its own, and in this case, there is a possibility that it may be mistaken for not being an object. To reduce the likelihood of misjudgment, artificial intelligence based on big data can be utilized. On the other hand, it is possible to reduce the incidence of judgment errors by using a general camera for imaging or by using a thermal imaging camera.

In one embodiment, the control unit may interlock with the seat belt module, and in particular, when the target object is an object, the control unit and the seat belt module may be interlocked so that the seat belt warning is not output.

Recently, a seat belt module that detects whether a seat belt is fastened and outputs a warning has been widely used and used. As an example, the seat belt module may include a weight sensing sensor provided in the seat and a sensor detecting whether the seat belt is fastened in the seat belt wearing mouth. Accordingly, when a seat belt is not fastened while an object having a predetermined reference weight or more is located in the seat, a warning sound or the like is output, and in the opposite case, a warning sound or the like is not output even if the seat belt is not fastened.

On the other hand, when an object that is not a person but has a weight that can be recognized as a person is placed in a seat, a problem in which a seat belt warning is not output may be generated simply according to the weight detection result. In addition, in this case, if the object placed on the seat is an object that cannot fasten the seat belt, a problem may occur in which a warning sound is outputted while driving.

However, according to an embodiment of the present invention, it is possible to determine whether the object placed on the seat is an object, and thus, if the target object is an object, it is possible to prevent the seat belt warning sound from being output despite the detection result of the weight sensor. .

In one embodiment, a memory unit that stores information about the shape and movement of a person's eyes, nose, mouth, ears, face, head, neck, shoulders, arms, chest, and stomach may be provided. Here, each information about a person can be classified into an adult, a child, and even an infant to be databased.

Accordingly, the control unit may determine whether the target object is a person by using information stored in the memory unit, and further, may distinguish whether the person is an adult, a child, or an infant.

Accordingly, when the target object is a person, a warning sound or a visual warning message may be output when the seat belt is not worn in conjunction with the seat belt module.

In addition, it is possible to determine whether the identified person is closing his eyes, breathing, or the like, and outputting the result of the determination as visual information or sound information through the output unit.

In one embodiment, the control unit may generate a command signal to output a warning that the seat belt is not worn on the output unit when the detected object is determined to be a creature. At this time, this command signal may be transmitted from the control unit to the output unit.

In one embodiment, the output unit may be implemented as a speaker or display originally mounted in a vehicle.

In another embodiment, the output unit may be implemented as a speaker or display that is additionally provided to enable proper warning according to a detection result of the vision sensor, even if it is not a vehicle-equipped speaker or display.

In one embodiment, a case may occur in which it is determined that the target object is not a person but is not an object. For example, this is the case when the target object is an animal such as a dog or a cat.

In one embodiment, the algorithm may be configured to determine that the object is an object if it is not a human, but in this case, there is a possibility that the object is wrong even if the object is an animal. In order to solve this problem, information about shapes and movements such as eyes, nose, mouth, ears, face, head, neck, shoulders, arms, chest, and stomach for common pets can be stored in the memory unit. have.

In another embodiment, instead of judging as an object when it is not a person or an animal, the characteristics of the object itself are stored in the memory unit, and data stored in the memory unit is used to determine whether the target object is an object or an object. You may.

The vehicle information device according to an embodiment of the present invention may further include at least one of a camera, a motion detection sensor, and a thermal imaging camera, and the control unit may include a plurality of operation units. For example, the first operation unit may perform a function of analyzing data acquired by the vision sensor, and the second operation unit may perform a function of analyzing data acquired by any one of a camera, a motion detection sensor, and a thermal imaging camera. Also, a third calculation unit that integrates calculation results of the first calculation unit and the second calculation unit may be further provided.

In one embodiment, as a result of analysis of the first calculation unit, when it is determined that the moving object in a predetermined seat is a person, the second calculation unit may process the image captured by the camera and display it as visual information or sound information to the driver. have.

In one embodiment, as a result of the analysis of the first operation unit, when it is determined that the moving object in the predetermined seat is a child, the second operation unit processes the image captured by the thermal imaging camera, and the third operation unit processes the first operation unit By interlocking the result and the processing result of the second operation unit, it is possible to determine whether there is an abnormality by analyzing the temperature distribution and respiration of the child.

The vehicle information device according to an embodiment of the present invention may further include a main server that controls at least some of matters related to the vehicle. In one embodiment, the main server may include at least one ECU that is basically mounted on a vehicle. In one embodiment, the main server may be connected to and interact with the vehicle utility. Here, the vehicle utility may include a seat belt module, a door lock module, a window lock module, a speaker module, a display module, and a shooting module.

In one embodiment, the control unit may notify the main server of the determination result, and the main server may control the aforementioned vehicle utility so that predetermined measures can be taken according to the determination result.

For example, when the determination result received from the control unit includes information that determines the target object as a living creature, the main server controls the seat belt module to activate the seat belt warning function, and in the determination result, the target object is determined to be inanimate If is included, the main server can disable the seat belt warning function.

In addition, when the determination result received from the control unit includes information that determines the target object as a child, the main server may generate a control signal to activate at least one of a door lock and a window lock of the vehicle. Here, when the door lock is activated, the corresponding door may not be opened even if the child operates the door open handle. In addition, when the window lock is activated, the window may not go down even if the child operates the window down button.

In one embodiment, the vision sensor may be located above the seat seat of the vehicle. For example, in FIG. 2, the first to fourth vision sensors 110-1, 110-2, 110-3, and 110-4 are vision sensors located on the seat seat. Meanwhile, the second vision sensor 110-2 illustrates a case installed in the headrest. And, referring to FIG. 3, an embodiment in which a first vision sensor is disposed on a second seat seat and the sensing range of the second vision sensor includes a fourth seat seat may be understood.

In one embodiment, the vision sensor may be located on the ceiling portion of the vehicle. In FIG. 2, the second display unit is an opening type display device, and the fifth vision sensor 110-5 can acquire data while the screen of the second display unit is open, and the sixth vision sensor opens or closes the screen of the second display unit. Data acquisition is possible regardless of. Meanwhile, referring to FIGS. 5 and 6, it may be understood how vision sensors (110-6, 110-8-1, 110-8-2) located in the ceiling portion of the vehicle include seat seats in the surveillance range. There will be.

In one embodiment, the vision sensor may be disposed in a center fascia or the like located on the front of the driver's seat of the vehicle, and in FIG. 2, an embodiment in which a seventh vision sensor is provided around the display screen is illustrated.

In one embodiment, the acquisition data of the entire plurality of vision sensors may be analyzed by integrating one controller.

In another embodiment, a plurality of control units may be provided, and each control unit may analyze acquisition data of one or more vision sensors.

Further, at least some of the vision sensors may be connected to the vehicle ECU, and the ECU may serve as the above-described controller.

In one embodiment, the vision sensor may be utilized to implement a rear occupant alert function. That is, when a child is asleep in a blind spot in a vehicle, the minute movement of the child can be determined by analyzing acquisition data of the vision sensor in which the corresponding seat is in the detection range. Accordingly, when a driver or the like leaves the vehicle in a state where the child is left unattended, a warning sound or a warning text may be transmitted to a related person. On the other hand, in the prior art, a technology for implementing the ROA function through an ultrasonic sensor has been introduced, but when the child is asleep and does not move, the ultrasonic sensor may not be able to detect it. When set, a problem may occur that a warning sound is generated even when luggage is placed in a predetermined seat. These limitations or problems in the ROA function using the ultrasonic sensor may be solved or supplemented by an embodiment of the present invention.

In one embodiment, through the data acquired by the vision sensor, when a movement or an object is detected at a predetermined position inside the vehicle, an additional verification procedure required according to the situation may be performed. For example, by operating a thermal imaging camera, measures such as determining whether an object in a blind spot is a person or an object may be taken.

Meanwhile, the detection range of the vision sensor may include the exterior of the vehicle. In this case, the vision sensor may be used for emergency braking assistance, parking assistance, and the like.

In one embodiment, the detection information pinhole may mean a vision sensor, a camera, a motion detection sensor, a thermal imaging camera, and the like.

1000: vehicle information device
100: detection information acquisition unit
110: vision sensor
120: camera
130: motion detection sensor
140: thermal imaging camera
200: control unit
300: memory unit
400: output
500: main server
2000: Automotive Utility
2100: seat belt module
2200: Door lock module
2300: Window lock module
2400: speaker module
2500: display module
2600: shooting module (for internal, external)

Claims (11)

A vision sensor provided in the vehicle; And
Includes; a control unit for analyzing the data acquired by the vision sensor,
The controller is a vehicle information device having a vision sensor, characterized in that for determining the object in the detection range of the vision sensor.
The method according to claim 1,
The control unit, the vehicle information device having a vision sensor, characterized in that for determining whether the object is living or inanimate.
The method according to claim 2,
The controller is a vehicle information device having a vision sensor, characterized in that to determine whether the creature is an adult or a child.
The method according to claim 2,
Further comprising an output connected to the control unit,
The controller is a vehicle information device having a vision sensor that generates a command signal to output a warning that the seat belt is not attached to the output unit when the object is determined to be a living thing.
The method according to claim 1,
Further comprising a camera, the control unit,
A first operation unit analyzing data acquired by the vision sensor;
A second operation unit to analyze the captured image of the camera; And
A third operation unit integrating the operation results of the first operation unit and the second operation unit;
Vehicle information device having a vision sensor comprising a.
The method according to claim 1,
Further comprising a main server for controlling at least some of the matters related to the vehicle,
The control unit notifies the determination result to the main server,
When the determination result includes information that determines the object as a creature, the main server activates a seat belt warning function,
When the determination result includes information that determines the object as inanimate, the main server has a vehicle information device having a vision sensor that deactivates the seat belt warning function.
The method according to claim 1,
Further comprising a main server for controlling at least some of the matters related to the vehicle,
The control unit notifies the determination result to the main server,
If the determination result includes information that determines the object as a child,
The main server is a vehicle information device having a vision sensor that generates a control signal so that at least one of the door lock and the window lock of the vehicle is activated.
The method according to claim 1,
The vision sensor is a vehicle information device having a vision sensor including a first vision sensor located on the ceiling of the vehicle.
The method according to claim 1,
The vision sensor is a vehicle information device having a vision sensor including a second vision sensor located on the seat seat of the vehicle.
The method according to claim 1,
A vehicle information device having a vision sensor further comprising a thermal imaging camera that photographs a thermal image of the object.
The method according to claim 10,
The control unit, the vehicle information device having a vision sensor to determine the state of the object by analyzing the results of the heat distribution of the object.

Images