KR102200218B1 - Camera device, system and method responsive to gaze direction of driver - Google Patents

Abstract

An imaging device that responds to the driver's gaze direction includes a receiver that receives motion information from a wearable device worn by the driver, a motion analysis section that analyzes the driver's motion based on the received motion information, and the image pickup device based on the analyzed motion. It includes a control unit for changing the imaging area, and when the motion analysis unit interprets that the movement corresponds to an accident, the control unit is configured to control the imaging device to take pictures of the surrounding situation and the driver while rotating the imaging area.

Description

An imaging device, an imaging system, and an imaging method that responds to the driver's gaze direction {CAMERA DEVICE, SYSTEM AND METHOD RESPONSIVE TO GAZE DIRECTION OF DRIVER}

The present invention relates to an image pickup device, an image pickup system, and an image pickup method that respond to the driver's line of sight.

The vehicle black box is called an automotive video accident recorder (EDR, Event Data Recorder), and is mainly used to cover the city and other ratios in the event of a traffic accident. Vehicle black boxes are mainly camera-type products, and are mainly installed around a room mirror or dashboard inside a vehicle to take an image of the front of the vehicle and record it as a video.

In relation to such a vehicle black box, Korean Patent Publication No. 2012-0026202, which is a prior art, discloses an image storage device for a vehicle.

In general, a vehicle black box is installed in a vehicle in the form of a fixed camera. However, since the camera is fixed, if an accident occurs in an area other than the shooting area of the camera, images related to the situation at the accident site cannot be obtained. In addition, in order to reduce such blind spots, side and rear cameras may be additionally installed on the vehicle, but it has a disadvantage of increasing cost. Therefore, there is a demand for a vehicle black box that can flexibly contain the situation of the accident site at low cost.

An image pickup device, an image pickup system, and an image pickup method that can acquire an image according to the driver's gaze in real time and respond to the driver's gaze direction are provided. In addition, in the event of an accident, the driver intends to provide an imaging device, an imaging system, and an imaging method that responds to the driver's gaze direction that can capture the accident situation in the side, rear, and front of the vehicle in real time through the wearable device. do. In addition, it is intended to provide an imaging device, an imaging system, and an imaging method that respond to the driver's gaze direction that can provide a stored image to a smartphone app through real-time streaming. However, the technical problem to be achieved by the present embodiment is not limited to the technical problems as described above, and other technical problems may exist.

As a technical means for achieving the above-described technical problem, an embodiment of the present invention provides a receiving unit for receiving motion information from a wearable device worn by a driver, a motion analyzing unit for analyzing the driver's motion based on the received motion information, and Includes a control unit that changes the imaging area of the imaging device based on the analyzed motion, but when the motion analysis unit interprets that the motion corresponds to an accident, the controller controls the imaging device to take pictures of the surrounding situation and the driver while rotating the imaging area. It is possible to provide an imaging device configured to be.

In addition, another embodiment of the present invention is a wearable device that is worn on the driver's body to generate motion information for the driver, and an image capturing that receives and analyzes motion information from the wearable device, and changes the imaging area based on the analyzed motion. Including a device, wherein the imaging device includes a cradle configured to enable pan and tilt operations, and when the imaging device interprets that movement corresponds to an accident, the imaging device rotates the imaging area while It is possible to provide an imaging system configured to control the surrounding situation and the driver to photograph.

In addition, another embodiment of the present invention includes receiving motion information from a wearable device worn by the driver, analyzing the driver's motion based on the received motion information, and capturing an imaging device based on the analyzed motion. Including the step of changing an area, and when it is interpreted that the movement corresponds to an accident, an imaging method configured to control the imaging device to take pictures of a surrounding situation and a driver while rotating the imaging area may be provided.

The above-described problem solving means are merely exemplary and should not be construed as limiting the present invention. In addition to the above-described exemplary embodiments, there may be additional embodiments described in the drawings and detailed description of the invention.

According to any one of the above-described problem solving means of the present invention, it is possible to obtain an image according to the driver's gaze in real time, and to provide an imaging device, an imaging system, and an imaging method that respond to the driver's gaze direction. In addition, in the event of an accident, the driver can provide an imaging device, an imaging system, and an imaging method that responds to the driver's gaze direction that can capture the accident situation in the side, rear, and front of the vehicle through the wearable device in real time. I can. In addition, it is possible to provide an imaging device, an imaging system, and an imaging method that responds to the driver's gaze direction that can provide a stored image to a smartphone app through real-time streaming.

1 is a block diagram of an imaging system that responds to a driver's gaze direction according to an embodiment of the present invention.
2 is a configuration diagram of an imaging device that responds to a driver's gaze direction according to an embodiment of the present invention.
3 is a flowchart illustrating a process of changing an imaging area in the imaging device according to an embodiment of the present invention.
4 is a flowchart illustrating an imaging method that responds to a driver's gaze direction according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, it means that other components may be further included, and one or more other features, not excluding other components, unless specifically stated to the contrary. It is to be understood that it does not preclude the presence or addition of any number, step, action, component, part, or combination thereof.

In the present specification, the term "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Further, one unit may be realized using two or more hardware, or two or more units may be realized using one hardware.

In the present specification, some of the operations or functions described as being performed by the terminal or device may be performed instead by a server connected to the terminal or device. Likewise, some of the operations or functions described as being performed by the server may also be performed by a terminal or device connected to the server.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of an imaging system that responds to a driver's gaze direction according to an embodiment of the present invention. Referring to FIG. 1, the imaging system 1 includes an imaging device 100, a wearable device 110, a mobile device 120, and a cloud server 130, and the imaging device 100 includes a camera 102 and a It may include a cradle 104. The imaging apparatus 100, the wearable device 110, the mobile device 120, and the cloud server 130 including the camera 102 and the cradle 104 shown in FIG. 1 are to be controlled by the imaging system 1. It shows the possible components by way of example.

Each component of the imaging system 1 of FIG. 1 is generally connected through a network. For example, as shown in FIG. 1, the wearable device 110 may be connected to the camera 102 or the cradle 104 through a network.

A network refers to a connection structure that enables information exchange between nodes such as terminals and servers, and examples of such networks include Wi-Fi, Bluetooth, Internet, and Local Area (LAN). Network), Wireless Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), 3G, 4G, LTE, etc., but are not limited thereto.

The imaging device 100 may include a camera 102 and a cradle 104 configured to enable pan and tilt operations. The cradle 120 may include a first actuator capable of panning and a second actuator capable of tilting. The first actuator may include a ring-type ultrasonic motor used for a camera lens, and the second actuator may include a vibration-type ultrasonic motor or a general micro-vibration motor.

The camera 102 may be combined with a wearable device 110 worn on the driver's body, for example, a coupling part of a Bluetooth headphone to photograph a driving situation according to the driver's gaze direction and store the captured image. have. In addition, the camera 102 is coupled with the cradle 104 mounted on the dashboard of the vehicle and communicates with the wearable device 110 worn on the driver's head, based on the movement of the driver's head, the rotation direction of the cradle 120 And it is possible to control the rotation angle.

The imaging device 100 may transmit an image captured through Wi-Fi or Bluetooth communication to the mobile device 120. Alternatively, the imaging device 100 may directly transmit an image captured by using various network communications to the cloud server 130 in real time.

In one embodiment, when the camera 102 is combined with the cradle 104 mounted on the dashboard of the vehicle, the imaging device 100 receives and interprets the driver's head movement information from the wearable device 110, and Through the control of the pan and tilt operations of 104, the imaging area of the camera 102 may be changed so that the rotation direction and the rotation angle of the cradle 104 correspond to the analyzed driver's head movement.

For example, when the imaging device 100 interprets that the driver's movement corresponds to an accident, the imaging device 100 rotates the imaging area of the camera 102 by 360° to capture the surrounding situation, the driver and the passenger seat status. It is possible to control the pan and tilt of the cradle 104 so that it can be performed.

For another example, when the imaging device 100 interprets that the accident is higher than or equal to a preset grade, it may be configured to transmit the captured image and location coordinates to a predetermined contact. At this time, the pre-set accident level includes Level 1: Safety, Level 2: Caution, Level 3: Vigilance, Level 4: Danger, Level 5: Very Danger, and if the pre-set accident level is 4 or higher, the captured image And location coordinates can be configured to transmit to the family, insurance company, 112 or 119.

For another example, when the acceleration sensor and the gyro sensor built in the wearable device 110 move vertically, the imaging device 100 interprets that the driver's movement corresponds to drowsiness, and the wearable device 110 Can be configured to output a warning message. In this case, the warning message may include a sleep prevention voice guidance, sleep prevention music, and sleep prevention vibration.

For another example, when the imaging device 100 interprets that the driver's head has rotated within a certain angle, it is determined that the driver's head movement is seldom, and the imaging area can be controlled to be fixed to face the front. have. For another example, when the imaging device 100 interprets that the driver's head has rotated by a certain angle or more, the imaging device 100 may control the pan and tilt of the cradle 104 to change the imaging area to the driver's head. It can be rotated according to the direction and angle of rotation. As another example, when the imaging device 100 interprets that the driver's head rotates by a certain angle or more and is maintained as it is, the imaging device 100 controls the image capture area through pan and tilt control of the cradle 104. After rotating according to the rotation of the driver's head, it can be rotated further.

The wearable device 110 may be worn on the driver's body to generate motion information about the driver. The wearable device 110 may transmit the generated motion information to the imaging apparatus 100 through Wi-Fi or Bluetooth communication. In this case, the wearable device 110 may include a neck back in which a battery is embedded and an ear set that may be in contact with a power supply terminal. In addition, the wearable device 110 may include an acceleration sensor and a gyro sensor to detect movement of the driver's head.

The mobile device 120 may be connected to the imaging apparatus 100 or the wearable device 110 to receive an image captured from the imaging apparatus 100 or the wearable device 110. In this case, the mobile device 120 may store the received image in a memory. Alternatively, the mobile device 120 may transmit an image received through an application installed in the mobile device 120 to the cloud server 130. In this case, the mobile device 120 may transmit GPS information about the location at which the image was captured to the cloud server 130 together.

The cloud server 130 may receive and store an image captured by the imaging apparatus 100 or the wearable device 110 from the mobile device 120. At this time, the cloud server 130 may record the location (eg, GPS record of the mobile device 120) captured of the received image through the GPS of the mobile device 120, and this Can be stored together. Alternatively, the cloud server 130 may directly receive and store an image captured from the imaging apparatus 100 or the wearable device 110. After passing through the authentication step, drivers can access the cloud server 130 to view the transmitted image.

2 is a configuration diagram of an imaging device that responds to a driver's gaze direction according to an embodiment of the present invention. Referring to FIG. 2, the imaging apparatus 100 may include a receiving unit 210, a motion analysis unit 220, and a control unit 230. In this case, the imaging apparatus 100 may include a camera 102 and a cradle 104 configured to enable pan and tilt operations.

The receiver 210 may receive motion information from the wearable device 110 worn by the driver. In this case, the receiver 210 may receive motion information through Wi-Fi or Bluetooth communication with, for example, a wearable device 110 such as a Bluetooth headset.

The motion analysis unit 220 may analyze the driver's motion based on the received motion information. For example, when an accident occurs with a vehicle other than the driver's vehicle and an impact is applied to the vehicle, the driver may cause severe movement in the neck joint. At this time, when the driver's neck joint moves severely, a large change occurs in the acceleration sensor and the gyro sensor value of the wearable device 110 worn by the driver, and the driver's movement can be analyzed based on the driver's movement information. In addition, the motion analysis unit 220 may interpret that the received motion information corresponds to an accident. In addition, when it is interpreted that the motion information received by the motion analysis unit 220 corresponds to an accident, the motion analysis unit 220 may analyze whether the accident is higher than a preset level based on the amount of impact of the accident. At this time, the accident level may include Level 1: Safety, Level 2: Caution, Level 3: Vigilance, Level 4: Danger, Level 5: Extremely Danger, and above the preset level is'Dangerous', which is level 4 of the impulse level. It can be a'very risky' step 5 and.

For another example, when the driver is drowsy driving, the wearable device 110 worn by the driver may move up and down. In this case, when the moving width of the wearable device 110 is large or the wearable device 110 continues to descend below a specific angle, the motion analysis unit 220 may interpret that the received motion information corresponds to drowsiness.

For another example, when the driver's vehicle enters an intersection with a large amount of movement of the vehicle, the probability of occurrence of an accident is increased, and thus the motion analysis unit 220 rotates the direction and rotation of the wearable device 110 worn by the driver. Motion information can be analyzed based on the angle. The motion analysis unit 220 rotates by determining whether the driver's head has rotated within a certain angle or whether the driver's head movement has rotated more than a certain angle based on the moving angle of the wearable device 110 worn by the driver. You can know the angle. In this case, when the driver's head movement rotates within a certain angle, the motion analysis unit 220 may consider that there is no movement of the driver. In addition, the motion analysis unit 220 may analyze whether the driver's head has been rotated by a certain angle or more and maintained as it is.

The controller 230 may change the imaging area of the imaging apparatus 100 based on the analyzed motion. The controller 230 may change the imaging area by controlling pan and tilt operations of the cradle 104. At this time, the cradle 104 may include a first actuator capable of panning and a second actuator capable of tilting, and the control unit 230 controls the operation of the first and second actuators of the cradle 104 can do.

For example, when the motion analysis unit 220 interprets that the driver's movement corresponds to an accident, the controller 230 rotates the imaging area by 360° so that the imaging device 100 photographs the surrounding situation and the driver. It can be configured to control. In this case, the control unit 230 may be configured to control the imaging device 100 to also photograph the state of the passenger seat.

For another example, when the motion analysis unit 220 interprets the accident as having a predetermined level or higher, the control unit 230 may be configured to transmit the captured image and the location coordinates acquired through GPS to a predetermined contact. . In this case, the accident level may include Level 1: Safety, Level 2: Caution, Level 3: Alert, Level 4: Danger, and Level 5: Very Danger. The controller 230 may automatically transmit the captured image and location coordinates to the family, insurance companies, 112 and 119 when the accident level is 4 or higher. Also, the controller 230 may upload the captured image and location coordinates to the cloud server in real time.

For another example, when the motion analysis unit 220 interprets that the driver's movement corresponds to drowsiness, the controller 230 may be configured to output a warning message to the wearable device 110 worn by the driver. . In this case, the warning message may include a sleep prevention voice guidance, sleep prevention music, and sleep prevention vibration.

For another example, when the motion analysis unit 220 interprets that the driver's head has rotated within a certain angle, it is considered that the driver's head movement is free, and the control unit 230 determines that the imaging area faces the front. It can be configured to fix it.

For another example, when the motion analysis unit 220 interprets that the driver's head has rotated by a predetermined angle or more, the controller 230 may be configured to rotate the imaging area according to the driver's head rotation. When the driver's vehicle enters a right turn at a crossroad, the driver will look behind the left to look in the left lane. At this time, the motion analysis unit 220 analyzes the movement of the driver's head rotating from front to left and rear, and the control unit 230 controls the cradle 104 in the same direction and angle as the driver's head rotation direction and rotation angle. can do. However, when another vehicle collides with the side of the driver's vehicle, the controller 230 detects the collision through a sensor and performs the operation of the cradle 104 according to the rotation direction and rotation angle based on the user's head movement. However, the fan rotation speed can be controlled to operate 2-3 times faster than the user's head movement.

For another example, if the motion analysis unit 220 interprets that the driver's head rotates by a certain angle or more and remains as it is, the controller 230 rotates the imaging area according to the driver's head rotation, and then additionally It can be configured to rotate. In this case, the motion analysis unit 220 may determine whether the driver's head has rotated sideways or vertically. For example, when the motion analysis unit 220 interprets that the driver's head rotates 10° to the left and then maintains it as it is, the controller 230 may rotate the pan 10° or more. In this case, when the driver rotates his/her head to the right, the controller 230 may stop the rotation of the fan. Alternatively, when the driver quickly rotates his/her head to the right, the controller 230 may control the camera of the cradle 104 to be positioned in the forward direction. For another example, when the driver's head moves to the top by about 5° in the motion analysis unit 220, the controller 230 moves the camera of the cradle 104 to the top by 5° and then stops it. can do. At this time, when the motion analysis unit 220 interprets that the driver's head has moved 10° in the upper direction and then maintained as it is, the control unit 230 may rotate the tilt by 10° or more. At this time, when the driver moves his head to the lower end quickly, the controller 230 may stop the rotation of the tilt.

3 is a flowchart illustrating a process of changing an imaging area in the imaging device according to an embodiment of the present invention. Referring to FIG. 3, the imaging apparatus 100 analyzes a driver's motion based on motion information received from the wearable device 110 (S310). The imaging device 100 determines whether the driver's head has rotated within a certain angle based on the analyzed driver's motion (S320). At this time, if the imaging device 100 determines that the driver's head movement has rotated within a certain angle (S321), the imaging device 100 may control the imaging area to face the front (S330). Alternatively, if the imaging device 100 determines that the driver's head movement is not rotated within a certain angle but has rotated more than a certain angle (S332), the imaging device 100 rotates the imaging area according to the rotation of the driver's head. It can be controlled to make it (S340).

In the above description, steps S310 to S340 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted as necessary, and the order between steps may be changed.

4 is a flowchart illustrating an imaging method that responds to a driver's gaze direction according to an embodiment of the present invention. The imaging method performed by the imaging device 100 responding to the driver's gaze direction according to the embodiment shown in FIG. 4 is in the imaging system 1 responding to the driver's gaze direction according to the embodiment shown in FIG. It includes steps that are processed in time series. Therefore, even if omitted below, the information already described about the imaging system 1 responding to the driver's gaze direction according to the embodiment shown in FIG. 1 is based on the driver’s gaze direction according to the embodiment shown in FIG. 4. It is also applied to the imaging method performed by the responsive imaging device 100.

In step S410, the imaging apparatus 100 receives motion information from the wearable device 110 worn by the driver. In step S420, the imaging device 100 analyzes the driver's motion based on the received motion information. In step S430, the imaging device 100 changes the imaging area of the imaging device 100 based on the analyzed motion. In this case, when it is interpreted that the movement corresponds to an accident, the imaging device 100 may be configured to control the imaging device 100 to capture the surrounding situation and the driver while rotating the imaging area.

In the above description, steps S410 to S430 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted as necessary, and the order between steps may be changed.

The imaging method performed by the imaging device 100 responsive to the driver's gaze direction described through FIG. 4 is also implemented in the form of a recording medium including instructions executable by a computer such as a program module executed by a computer. Can be. Computer-readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Further, the computer-readable medium may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transmission mechanism, and includes any information delivery media.

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains will be able to understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative and non-limiting in all respects. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

100: image pickup device
102: camera
104: cradle
110: wearable device
120: mobile device
130: cloud server
210: receiver
220: motion analysis unit
230: control unit

Claims (20)

In the imaging device that responds to the driver's gaze direction,
A receiver configured to receive motion information from the wearable device worn by the driver;
A motion analysis unit that analyzes the driver's motion based on the received motion information; And
A control unit for changing the imaging area of the imaging device based on the analyzed motion;
Including,
When the motion analysis unit interprets that the motion corresponds to drowsiness,
To the wearable device configured to output a warning message.
The method of claim 1,
Wherein the control unit is configured to control the imaging device to capture a state of a passenger seat.
The method of claim 1,
When the motion analysis unit interprets the accident as having a preset rating or higher,
The control unit is configured to transmit the captured image and position coordinates to a predetermined contact information.
The method of claim 1,
When the motion analysis unit interprets that the motion corresponds to an accident,
The control unit is configured to control the imaging device to take pictures of a surrounding situation and the driver while rotating the imaging area.
The method of claim 1,
When the motion analysis unit interprets that the driver's head has rotated within a certain angle,
Wherein the control unit is configured to fix the imaging area to face the front side.
The method of claim 1,
When the motion analysis unit interprets that the driver's head has rotated more than a certain angle,
The control unit is configured to rotate the imaging area according to the rotation of the driver's head.
The method of claim 6,
When the motion analysis unit interprets that the driver's head rotated by a certain angle or more and maintained it
The control unit is configured to further rotate the imaging area after rotating according to the rotation of the driver's head.
The method of claim 1,
Further comprising a vehicle cradle configured to enable pan and tilt operation,
The control unit is configured to control the operation of the cradle.
The method of claim 8,
The cradle includes a first actuator capable of panning and a second actuator capable of tilting.
The method of claim 1,
The imaging device is configured to be mounted on the wearable device.
In an imaging system that responds to the driver's gaze direction,
A wearable device that is worn on the driver's body and generates motion information about the driver;
An imaging device that receives and analyzes the motion information from the wearable device, and changes an imaging area based on the analyzed motion;
Including,
The imaging device includes a cradle configured to enable pan and tilt operations;
Including,
When the imaging device interprets that the movement corresponds to drowsiness,
And outputting a warning message to the wearable device.
The method of claim 11,
Wherein the imaging device is configured to control the state of the passenger seat to be photographed.
The method of claim 11,
When the imaging device interprets that the accident is above a preset rating,
Is configured to transmit the captured image and location coordinates to a predetermined contact, imaging system.
The method of claim 11,
When the imaging device interprets that the movement corresponds to an accident,
An imaging system, configured to control the imaging device to take pictures of a surrounding situation and the driver while rotating the imaging area.
The method of claim 11,
When the imaging device interprets that the driver's head has rotated within a certain angle,
An imaging system configured to fix the imaging area to face the front side.
The method of claim 11,
When the imaging device interprets that the driver's head has rotated by a certain angle or more,
And the imaging system being configured to rotate the imaging area according to the rotation of the driver's head.
The method of claim 16,
When the imaging device is interpreted as maintaining the driver's head after rotating at a certain angle or more,
The imaging system is configured to further rotate the imaging area after rotating according to the rotation of the driver's head.
In an imaging system that responds to the driver's gaze direction,
A wearable device that is worn on the driver's body and generates motion information about the driver;
An imaging device that receives and analyzes the motion information from the wearable device, and changes an imaging area based on the analyzed motion;
Including,
The imaging device includes a cradle configured to enable pan and tilt operations;
Including,
The imaging device is configured to control the operation of the cradle,
The cradle includes a first actuator capable of panning and a second actuator capable of tilting.
The method of claim 11,
Wherein the imaging device is configured to be mounted on the wearable device.
In the imaging method that responds to the driver's gaze direction,
Receiving motion information from a wearable device worn by the driver;
Analyzing the driver's motion based on the received motion information; And
Changing an imaging area of the imaging device based on the analyzed motion;
Including,
If the movement is interpreted as drowsiness
Further comprising a step, configured to output a warning message to the wearable device.

Images