KR20210045370A - Vehicle information device having vision sensor - Google Patents

Abstract

According to an exemplary embodiment of the present invention, a vehicle information device having a vision sensor includes: a vision sensor provided in a vehicle; and a control unit that analyzes the data acquired by the vision sensor, wherein the control unit may determine an object within a sensing range of the vision sensor. The present invention can improve the monitoring efficiency of the vehicle situation.

Description

Vehicle information device with vision sensor {VEHICLE INFORMATION DEVICE HAVING VISION SENSOR}

An embodiment of the present invention relates to a vehicle information device having a vision sensor.

A technology for detecting various motions or gestures using a vision sensor has been developed, and research to achieve various purposes using this technology is ongoing.

An aspect of the present invention can provide a vehicle information device having a vision sensor capable of improving the monitoring efficiency of a vehicle condition.

A vehicle information device having a vision sensor according to an exemplary embodiment of the present invention includes: a vision sensor provided in a vehicle; And a control unit for analyzing data acquired by the vision sensor, wherein the control unit may determine an object within a detection range of the vision sensor.

In this case, the controller may determine whether the object is a living thing or an inanimate object.

In addition, the controller may determine whether the creature is an adult or a child.

In addition, an output unit connected to the control unit may be further included, and the control unit may generate a command signal for outputting a warning not to wear a seat belt to the output unit when the object is determined to be a living creature.

Further, further comprising a camera, wherein the control unit, the first operation unit for analyzing the data acquired by the vision sensor; A second operation unit that analyzes the captured image of the camera; And a third operation unit for integrating the operation result of the first operation unit and the second operation unit.

Further, it further comprises a main server for controlling at least a part of the matters related to the vehicle, the control unit notifies the determination result to the main server, and if the determination result includes information that determines the object as a living organism, the main The server activates the seat belt non-wearing warning function, and when the determination result includes information on which the object is determined to be inanimate, the main server may deactivate the seat belt non-wearing warning function.

In addition, it further comprises a main server for controlling at least a part of the matters related to the vehicle, the control unit notifies the determination result to the main server, and if the determination result includes information that determines the object as a child, the main The server may generate a control signal to activate at least one of a door lock and a window lock of the vehicle.

In addition, the vision sensor may include a first vision sensor positioned on the ceiling of the vehicle.

In addition, the vision sensor may include a second vision sensor positioned above the seat seat of the vehicle.

In addition, it may further include a thermal imaging camera for taking a thermal image of the object.

In addition, the controller may determine the state of the object by analyzing the result of the heat distribution of the object.

According to an embodiment of the present invention, it provides a useful effect that the monitoring efficiency of the vehicle condition can be improved.

1 is a block diagram schematically illustrating a vehicle information device having a vision sensor according to an embodiment of the present invention,
2 is a view for explaining an arrangement position of a vision sensor in a vehicle according to an embodiment of the present invention,
3 is a view for explaining the position and detection range of the vision sensor according to an embodiment of the present invention,
4 is a view for explaining the position and detection range of a vision sensor according to another embodiment of the present invention,
5 is a diagram schematically illustrating a view of the embodiment of FIG. 4 from a different angle,
6 is a modified example of the embodiment illustrated in FIG. 5.

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, and may be implemented in various different forms. The present embodiments may be provided to complete the disclosure of the present invention and to fully inform the scope of the present invention to those of ordinary skill in the art to which the present invention pertains. The same reference numerals refer to the same components throughout the entire specification.

The terms used in the present specification are for describing exemplary embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase. As used in the specification,'comprise' and/or'comprising' refers to the presence of one or more other elements, steps, actions and/or elements, and/or elements, steps, actions and/or elements mentioned. Or does not preclude additions.

Further, the embodiments described in the present specification will be described with reference to sectional views and/or plan views, which are ideal exemplary diagrams of the present invention. In the drawings, detailed sizes, shapes, thicknesses, curvatures, etc. of each component are exaggerated or schematic for effective description of technical contents, and their shape may be modified due to tolerances.

Hereinafter, the configuration and operation of the present invention will be described in more detail with reference to the accompanying drawings.

1 is a block diagram schematically illustrating a vehicle information device having a vision sensor according to an embodiment of the present invention, and FIG. 2 is a view for explaining an arrangement position of a vision sensor in a vehicle according to an embodiment of the present invention. 3 is a view for explaining the position and detection range of a vision sensor according to an embodiment of the present invention, Figure 4 is a view for explaining the position and detection range of the vision sensor according to another embodiment of the present invention 5 is a diagram schematically illustrating a view of the embodiment of FIG. 4 from a different angle, and FIG. 6 is a modified example of the embodiment illustrated in FIG. 5.

Referring to the drawings, the vehicle information apparatus 1000 according to an embodiment of the present invention may include a vision sensor 110 and a control unit 200 to determine an object OBJ.

Here, determining the object OBJ may mean determining whether the object OBJ is an object, a human, or an animal.

* Further, it may mean to determine whether an operation performed by the object OBJ corresponds to a predetermined predetermined operation.

In one embodiment, the vision sensor 110 may be a device that performs a function of generating at least one event signal by detecting at least a part of the object OBJ in which movement has occurred.

In one embodiment, the vision sensor 110 may include a plurality of sensing elements, and each sensing element may detect the occurrence of a predetermined event and output an event signal.

In one embodiment, the vision sensor 110 may include at least one of an event-based vision sensor and a frame-based vision sensor. Here, the event-based vision sensor may generate an event signal asynchronously in time in response to an event in which light received from the object OBJ changes. In one embodiment, the event-based vision sensor may include a Dynamic Vision Sensor (DVS). Dynamic vision sensors respond only to changes in light, and dynamic vision sensors can detect the contours of moving objects. Accordingly, since a predetermined purpose can be realized even when a relatively small amount of data is processed, a target can be detected relatively quickly even when a relatively small amount of resources is used.

On the other hand, it is also possible to grasp the outline of the stationary object OBJ by moving or vibrating the dynamic vision sensor itself in a predetermined manner.

In one embodiment, the frame-based vision sensor may be a sensor that photographs an object (OBJ) synchronously according to a preset number of frames per second, and may include a CMOS image sensor (CIS).

The above-described event-based vision sensor only needs to use the address of the sensing element or the address and time information of the sensing element that detects the event in which the light has changed, so the amount of computation can be reduced and the power consumption can be reduced. It can have an advantage over vision sensors.

In one embodiment, the vision sensor 110 may acquire information on an object (OBJ) located within a sensing range (eg, field of view (FOV)) and provide it to the control unit 200. In addition, the controller 200 may analyze data provided from the vision sensor 110 to determine whether the object OBJ is an object, an adult, or a child.

In one embodiment, the detection range of the vision sensor 110 may include respective seats disposed inside the vehicle.

In one embodiment, a plurality of seats may be included in the sensing range of one vision sensor 110. For example, as illustrated in FIG. 5, a third seat may be included in the sensing range of the sixth vision sensor 110-6. (SEAT3) and the fourth sheet (SEAT4) can be included.

In another embodiment, vision sensors that are individually matched for each of the seats may be provided. For example, as illustrated in FIG. 6, the third sheet SEAT3 is included in the detection range of the 8-1 vision sensor 110-8-1, and the 8-2 vision sensor 110-8-2 The fourth sheet SEAT4 may be included in the detection range of.

In one embodiment, the controller 200 may determine that the object OBJ located in a specific seat is an object by analyzing information provided from the vision sensor 110. At this time, in order to determine whether the target object OBJ is an object, it is possible to determine whether the outline of the object OBJ is deformed or whether the outline of the object OBJ is moving. In one embodiment, when it is determined that the time asynchronous transformation or movement of the outline of the target object OBJ and the outline of the target object OBJ does not belong to a living creature, it may be determined that the target object OBJ is an object. However, the object object (OBJ) may be a toy doll that can move by itself, and in this case, there is a possibility that it is misjudged as not an object. In order to reduce the possibility of such misjudgment, big data-based artificial intelligence can be used. On the other hand, it is also possible to use the general camera 120 for image capturing or the thermal imaging camera 140 to reduce the rate of occurrence of a judgment error.

In one embodiment, the control unit 200 may be interlocked with the seat belt module 2100, and in particular, the control unit and the seat belt module 2100 are configured so that a warning not to be worn seat belt is not output when the target object OBJ is an object. It can be linked.

Recently, a seat belt module that detects whether a seat belt is fastened and outputs a warning has been widely used and utilized. As an example, the seat belt module may include a weight sensor provided in a seat and a sensor for detecting whether or not a seat belt is fastened. Accordingly, when the seat belt is not fastened while the object OBJ having a predetermined reference weight or more is positioned on the seat, a warning sound is output, and in the opposite case, a warning sound is not output even if the seat belt is not fastened.

On the other hand, when an object that is not a person but has a weight enough to be recognized as a person is placed on the seat, a problem that a warning for not wearing a seat belt is simply outputted according to the weight detection result may occur. In addition, in this case, if the object placed on the seat is an object that cannot be fastened with a seat belt, a warning sound may be output while driving.

However, according to an embodiment of the present invention, it is possible to determine whether the object OBJ placed on the seat is an object, and therefore, if the object OBJ is an object, the seat belt warning sound, etc., is generated despite the detection result of the weight sensor. You can prevent it from being printed.

In one embodiment, a memory unit 300 storing information on the shape and movement of a person's eyes, nose, mouth, ears, face, head, neck, shoulders, arms, chest, belly, etc. may be provided. Here, each information on a person can be classified into an adult, a child, and furthermore, an infant, and converted into a database.

Accordingly, the control unit 200 may determine whether the target object OBJ is a person, using the information stored in the memory unit 300, and furthermore, whether the person is an adult, a child, or an infant. You can also distinguish.

Accordingly, when the target object OBJ is a person, it is interlocked with the seat belt module 2100 to output a warning sound and a visual warning message when the seat belt is not worn.

In addition, it may be determined whether the identified person is closing their eyes or breathing, and the determination result may be output as visual information or sound information through the output unit 400.

In one embodiment, when the detected object OBJ is determined to be a living creature, the controller may generate a command signal to output a warning that the seat belt is not worn on the output unit 400. At this time, this command signal may be transmitted from the control unit 200 to the output unit 400.

In one embodiment, the output unit 400 may be implemented as a speaker or a display originally installed in the vehicle.

In another embodiment, the output unit may be implemented as a speaker or display additionally provided to enable an appropriate warning or the like according to the detection result of the vision sensor 110, even if it is not a basic speaker or display of the vehicle.

In one embodiment, there may be a case where it is determined that the target object OBJ is not a person but is not an object. For example, this is the case when the target object OBJ is an animal such as a dog or a cat.

In one embodiment, an algorithm may be configured to determine that the target object OBJ is an object if it is not a person. In this case, even if the target object OBJ is an animal, there is a possibility that it is misjudged as an object. In order to solve this problem, information on shapes and movements such as eyes, nose, mouth, ears, face, head, neck, shoulders, arms, chest, belly, etc. for common pets are also stored in the memory unit 300 You can do it.

In another embodiment, when it is not a person or an animal, the object is not determined as an object, but the characteristics of the object are stored in the memory unit 300, and the data stored in the memory unit 300 is used as a target object (OBJ). It can also be made to determine if it is an object.

The vehicle information device 1000 according to an embodiment of the present invention may further include at least one of a camera 120, a motion detection sensor 130, and a thermal imaging camera 140, and the controller 200 It may include an operation unit. For example, the first operation unit performs a function of analyzing data acquired by the vision sensor 110, and the second operation unit performs a function of analyzing data obtained by any one of a camera, a motion detection sensor, and a thermal imaging camera. I can. In addition, a third operation unit for integrating the operation result of the first operation unit and the second operation unit may be further provided.

In one embodiment, as a result of analysis of the first operation unit, when it is determined that the object OBJ moving in a predetermined seat is a person, the second operation unit processes the image captured by the camera and displays it as visual information or as acoustic information. Can be displayed.

In one embodiment, as a result of analysis of the first operation unit, when it is determined that the object OBJ moving in a predetermined seat is a child, the second operation unit processes the image captured by the thermal imaging camera, and the third operation unit processes the first operation unit. By interlocking the processing result of the operation unit and the processing result of the second operation unit, the child's body temperature distribution and respiration may be analyzed to determine whether there are any abnormal symptoms.

The vehicle information device 1000 according to an embodiment of the present invention may further include a main server 500 that controls at least some of the vehicle-related matters. In one embodiment, the main server 500 may include at least one ECU that is basically mounted on a vehicle. In one embodiment, the main server 500 may be connected to and interact with the vehicle utility 2000. Here, the vehicle utility 2000 may include a seat belt module 2100, a door lock module 2200, a window lock module 2300, a speaker module 2400, a display module 2500, a photographing module 2600, and the like.

In one embodiment, the control unit 200 may notify the determination result to the main server 500, and the main server 500 controls the above-described vehicle utility 2000 so that predetermined actions can be taken according to the determination result. can do.

For example, if the determination result provided from the control unit 200 includes information that determines the target object OBJ as a living creature, the main server 500 controls the seat belt module 2100 to activate a warning function for not wearing a seat belt. , If the determination result includes information on which the target object OBJ is determined to be inanimate, the main server 500 may deactivate the warning function of not wearing a seat belt.

In addition, if the determination result provided from the control unit 200 includes information that determines the target object OBJ as a child, the main server 500 generates a control signal to activate at least one of a door lock and a window lock of the vehicle. I can. Here, when the door lock is activated, the door may not be opened even if the child manipulates the door open handle. In addition, when the window lock is activated, the window may not go down even if the child operates the window down button.

In one embodiment, the vision sensor 110 may be located above the seat seat of the vehicle. For example, in FIG. 2, the first to fourth vision sensors 110-1, 110-2, 110-3 and 110-4 are vision sensors located above the seat seat. Meanwhile, the second vision sensor 110-2 exemplifies a case where it is installed on the headrest. And, referring to Figure 3, the first vision sensor (110-1) is disposed on the second seat seat (SEAT2), the detection range of the first vision sensor (110-1) is the fourth seat seat (SEAT4) It will be understood that an embodiment including a.

In one embodiment, the vision sensor may be located in the ceiling portion of the vehicle. In FIG. 2, the second display unit DP2 is an opening/closing display device, and the fifth vision sensor 110-5 can acquire data while the screen of the second display unit is open, and the sixth vision sensor 110-6 ) Can acquire data regardless of whether the screen of the second display unit DP2 is opened or closed. Meanwhile, referring to FIGS. 5 and 6, it can be understood how the vision sensors 110-6, 110-8-1, and 110-8-2 located on the ceiling of the vehicle include seat seats in the monitoring range. There will be.

In one embodiment, the vision sensor may be disposed on a center fascia located in front of a driver's seat of a vehicle, and FIG. 2 illustrates an embodiment in which the seventh vision sensor 110-7 is provided around the display screen DP1. .

In one embodiment, the acquired data of all of the plurality of vision sensors may be integrated and analyzed by one control unit 200.

In another embodiment, a plurality of controllers may be provided, and each controller may analyze acquired data of one or more vision sensors.

In addition, at least some of the vision sensors may be connected to an ECU for a vehicle, and the ECU may perform the role of the above-described control unit.

In one embodiment, the vision sensor may be used to implement a rear occupant alert function. That is, if the child is asleep in the blind spot in the vehicle, by analyzing the acquired data of the vision sensor that the seat is in the detection range, it is possible to determine the child's fine movement. Accordingly, when a driver or the like leaves the vehicle while leaving the child unattended, a warning sound or a warning text can be transmitted to the relevant person. On the other hand, conventionally, a technology for implementing the ROA function through an ultrasonic sensor has also been introduced. However, when a child is deeply asleep and does not move, the ultrasonic sensor may not detect it. Conversely, the ultrasonic sensor may detect only distance. If set, there may be a problem that a warning sound is generated even when luggage is placed in a predetermined seat. In the ROA function using an ultrasonic sensor, these limitations or problems can be solved or supplemented by an embodiment of the present invention.

In an embodiment, when a motion or object OBJ is detected at a predetermined position inside the vehicle through data acquired by the vision sensor, an additional confirmation procedure required according to the situation may be performed. For example, by operating the thermal imaging camera 140, it is possible to take measures such as determining whether the object OBJ in the blind spot is a person or an object.

Meanwhile, the detection range of the vision sensor may include the outside of the vehicle. In this case, the vision sensor may be used for emergency braking assistance, parking assistance, and the like.

In one embodiment, the detection information acquisition unit 100 may mean a vision sensor 110, a camera 120, a motion detection sensor 130, a thermal imaging camera 140, and the like.

1000: vehicle information device
100: detection information acquisition unit
110: vision sensor
120: camera
130: motion detection sensor
140: thermal imager
200: control unit
300: memory unit
400: output
500: main server
2000: vehicle utility
2100: seat belt module
2200: door lock module
2300: Window lock module
2400: speaker module
2500: display module
2600: shooting module (internal, external)

Claims (5)

A vision sensor provided in the vehicle; And
Including; a control unit for analyzing the data acquired by the vision sensor,
The vision sensor generates an event signal asynchronously in time in response to an event in which light received from an object changes, and uses the address and time information of the sensing element that detects the light change event to determine the outline of the object. Is a dynamic vision sensor that detects,
The control unit determines an object in the detection range of the vision sensor,
Includes; a first operation unit for analyzing the data acquired by the vision sensor,
And the control unit determines whether the object is a living thing or an inanimate object by analyzing whether the time-synchronized transformation of the contour of the object is a transformation of the contour of a living thing.
The method according to claim 1,
Further comprising an output unit connected to the control unit,
The control unit generates a command signal for outputting a warning that a seat belt is not worn on the output unit when the object is determined to be a living thing, and the control unit prevents the warning that a seat belt not be worn on the output unit when the object is determined to be inanimate Vehicle information device having a vision sensor, characterized in that.
The method according to claim 1,
Further comprising at least one of a camera, a motion detection sensor, and a thermal imaging camera,
The control unit,
A second operation unit that analyzes the captured image of the camera; And
A third operation unit for integrating operation results of the first operation unit and the second operation unit;
Further comprising, the third operation unit,
Vehicle information having a vision sensor, characterized in that, even when a change in the object contour is detected by the first operation unit, the analysis result of the second operation unit is collected to determine whether the object is a living thing or an inanimate object. Device.
The method according to claim 1,
Further comprising a main server for controlling at least some of the matters related to the vehicle,
The control unit notifies the determination result to the main server,
When the determination result includes information that determines the object as a living creature, the main server activates a safety belt not worn warning function,
When the determination result includes information on which the object is determined to be inanimate, the main server has a vision sensor for deactivating a warning function of not wearing a seat belt.
The method of claim 3,
The control unit is a vehicle information device having a vision sensor to determine the state of the object by analyzing the result of the heat distribution of the object.

Images