KR20220105810A - Control system using gesture and control method thereof - Google Patents

Abstract

The present invention relates to a vehicle control system using a gesture and a control method therefor comprising: a gesture recognition part equipped in a mission vehicle to detect a gesture of an operator; and a control part that controls driving of the mission vehicle according to the gesture of the operator detected in the gesture recognition part, wherein the gesture recognition part may be equipped with a priority gesture for a specific operator among a plurality of operators to acquire control rights of driving of the mission vehicle. In addition, the vehicle control system and the control method therefor using the same gesture as described above may be variously implemented according to the embodiments. Therefore, the present invention is capable of having an advantage of enabling proximity operation.

Description

A vehicle control system using a gesture and a control method thereof

The present invention relates to a vehicle control system using a gesture and a control method thereof, and to a vehicle control system and a control method thereof using a gesture capable of controlling various operations of a mission vehicle by sensing a user's hand.

In general, in the case of a vehicle that performs a mission, for example, a mission vehicle such as a wireless robot, the operator rides directly and drives manually, or remotely using a station and a portable control device such as an externally controlled vehicle, or setting a target point Thus, it can be driven by autonomous return driving that implements autonomous return.

However, in the case of the driving of such a mission vehicle, when the operation is performed at a short distance, there is a possibility that light is generated on the screen of the remote control device and the position of the operator is grasped. In particular, when the mission vehicle has a military target, various military risks may occur when the location of the operator is identified.

Therefore, when controlling the driving of the mission vehicle, the operator's position is not detected through the vision recognition of the operator through the camera provided in the mission vehicle, not the remote control device, using a gesture capable of controlling the operation of the mission vehicle. There is a need for a vehicle control system and a control method thereof.

US Registered Patent No. US10459529 (2019.10.29)

The technical problem to be achieved by the present invention is that it is possible to control the operation of a mission vehicle through vision recognition captured by a camera of the mission vehicle, and accordingly, a vehicle control system using a gesture to prevent the operator's position from being detected, and a control method thereof would like to provide

In addition, an object of the present invention is to provide a vehicle control system and a control method thereof using a gesture for obtaining control rights without confusion even in the presence of a plurality of operators and thus enabling operation of a mission vehicle.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

A vehicle control system using a gesture according to an embodiment of the present invention comprises:

a gesture recognition unit provided in the mission vehicle to detect an operator's gesture; and

a control unit for controlling driving of the mission vehicle according to the gesture of the operator detected by the gesture recognition unit;

The gesture recognition unit may be provided with a priority gesture for a specific operator among a plurality of operators to obtain a control right to drive the mission vehicle.

The gesture recognition unit may be operated in at least one of a vision recognition mode for detecting a gesture of the operator through a vision camera sensor, and a motion communication mode for detecting a motion of a motion recognition glove mounted on the operator.

The gesture recognition unit may include a forward gesture, a backward gesture, a stop gesture, a steering gesture, and an end gesture for terminating operation of the gesture along with the priority gesture for acquiring control.

In addition, the method for controlling a vehicle using a gesture according to an embodiment of the present invention includes: obtaining, by an operator, a control right of a mission vehicle;

recognizing a pattern of a gesture of the operator through a gesture recognition unit of the mission vehicle;

inquiring a vehicle control command corresponding to the gesture pattern; and

and controlling the operation of the mission vehicle according to the vehicle control command.

The gesture recognition unit may detect the gesture of the operator through the vision camera sensor, or detect the motion of the motion recognition glove mounted on the operator.

When there are a plurality of operators, the acquisition of the control right may be determined according to a priority gesture for acquiring the priority control right of driving the mission vehicle.

The details of other embodiments are included in the detailed description and drawings.

In the vehicle control system and control method thereof using a gesture according to the embodiment of the present invention as described above, if the camera mounted on the mission vehicle recognizes the gesture of the operator spaced apart from the mission vehicle, the position of the operator is not detected. It has the advantage of enabling close operation.

In addition, the vehicle control system using a gesture and a control method thereof according to an embodiment of the present invention have the advantage of recognizing a gesture without confusion through acquisition of the control right of the main operator even in the presence of a plurality of operators.

The effects of the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

1 is a schematic configuration diagram of a vehicle control system using a gesture according to an embodiment of the present invention.
2 is a diagram illustrating a gesture recognition unit in a vehicle control system using a gesture according to an embodiment of the present invention.
3 is a diagram illustrating a gesture corresponding to operation of a mission vehicle in a vehicle control system using a gesture according to an embodiment of the present invention.
4 is a diagram schematically illustrating acquisition of a control right in a vehicle control system using a gesture according to an embodiment of the present invention.
5 is a flowchart schematically illustrating a vehicle control method using a gesture according to an embodiment of the present invention.
6 is a control flowchart schematically illustrating a vehicle control method using a gesture according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but can be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

Accordingly, in some embodiments, well-known process steps, well-known structures, and well-known techniques have not been specifically described in order to avoid obscuring the present invention.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, the singular also includes the plural, unless specifically stated otherwise in the phrase. As used herein, includes and/or comprising refers to the presence or addition of one or more other components, steps, operations and/or elements other than the recited elements, steps, operations and/or elements. It is used in the sense of not being excluded. And, “and/or” includes each and every combination of one or more of the recited items.

In addition, the embodiments described herein will be described with reference to cross-sectional and/or schematic diagrams that are ideal illustrative views of the present invention. Accordingly, the shape of the illustrative drawing may be modified due to manufacturing technology and/or tolerance. Accordingly, the embodiments of the present invention are not limited to the specific form shown, but also include changes in the form generated according to the manufacturing process. In addition, in each of the drawings shown in the present invention, each component may be enlarged or reduced to some extent in consideration of convenience of description. Like reference numerals refer to like elements throughout.

Hereinafter, the present invention will be described with reference to the drawings for explaining the vehicle control system 100 using a gesture and a control method thereof according to embodiments of the present invention.

1 is a schematic configuration diagram of a vehicle control system 100 using a gesture according to an embodiment of the present invention.

2 is a diagram illustrating the gesture recognition unit 110 in the vehicle control system 100 using a gesture according to an embodiment of the present invention.

3 is a diagram illustrating a gesture corresponding to operation of the mission vehicle 20 in the vehicle control system 100 using a gesture according to an embodiment of the present invention.

4 is a diagram schematically illustrating acquisition of a control right in the vehicle control system 100 using a gesture according to an embodiment of the present invention.

1 to 4 , a vehicle system using a gesture according to an embodiment of the present invention may include a gesture recognition unit 110 and a control unit 120 .

Specifically, the gesture recognition unit 110 may be provided in the mission vehicle 20 to detect the operator's gesture 10 .

When the operator takes a set gesture with one hand or both hands, the gesture recognition unit 110 detects the operator's gesture 10 , and the mission vehicle 20 responds to the operator's gesture 10 . operation will be driven.

In addition, the control unit 120 may receive the signal detected by the gesture recognition unit 110 and control the mission vehicle 20 to be driven in response to the signal applied according to the gesture 10 of the operator. have.

First, the gesture recognition unit 110 is provided with at least one of a vision camera sensor 111 or a motion communication unit 112 to detect the operator's gesture 10, so as to detect the operator's gesture 10. can For example, when the gesture recognition unit 110 is provided as the vision camera sensor 111 , the vision camera sensor 111 implements vision recognition for detecting the shape of at least one hand of the operator, The mission vehicle 20 may be operated according to vision recognition (vision recognition mode). Alternatively, when the gesture recognition unit 110 is provided as the motion communication unit 112 , the operator may be provided with a motion recognition glove 10a capable of wirelessly transmitting and receiving with the motion communication unit 112 . Accordingly, the motion communication unit 112 may receive the motion form of the motion recognition glove 10a worn by the operator through wireless communication with the motion recognition glove 10a (motion communication mode). The received signal is applied to the control unit 120 , and the control unit 120 controls the mission vehicle 20 , and the mission vehicle 20 may be operated according to the operator's gesture 10 .

As described above, the gesture recognition unit 110 includes a priority gesture 11, a forward gesture 12, a backward gesture 12, and a stop gesture 14 for obtaining control, moving forward, backward, stopping, steering, and changing the operation method. , a steering gesture 15 and an end gesture 16 .

The priority gesture 11 is a gesture for obtaining priority control right of driving the mission vehicle 20 when there are a plurality of operators. The priority gesture 11 may be set as, for example, an OK sign type gesture by interlocking the operator's thumb and index finger. Accordingly, a priority gesture 11 is provided among the plurality of operators. Accordingly, when a specific operator among a plurality of operators takes the priority gesture 11 , the gesture recognition unit 110 sets the priority control right to preferentially detect the motion of the specific operator who has taken the priority gesture 11 . Accordingly, the gesture recognition unit 110 can operate the mission vehicle 20 by detecting the motion of a specific operator who has obtained priority control from among a plurality of operators.

In addition, the order of obtaining control rights may be set in the gesture recognition unit 110 in preparation for application of a plurality of priority gestures 11 signals.

For example, when the priority gesture 11 is applied in several places, the priority gesture 11 of the highest among the operators is set to have the highest priority, and then the priority gesture 11 may be set in the order of the highest priority and the first requested.

Accordingly, when a signal of the acquisition priority gesture 11 having a high priority is applied during operation, the control unit 120 can transfer the control right to the operator who performed the priority gesture 11 having a high priority.

For example, when the superlative person executes the priority gesture 11 during operation of a specific operator, the control unit 120 receives the superlative priority gesture 11 signal and transfers the priority control to the superlative person, and the mission vehicle ( 20) can be driven through the gesture of the superlative.

In addition, the control right acquisition order may be separately set by the operator. For example, the operator may set his/her left side as the first operator and the right side as the second operator. Accordingly, when the first operator performs the priority gesture 11 while operating the mission vehicle 20 by the operator and a signal is applied to the gesture recognition unit 110, the control unit 120 controls the first operation according to the applied signal. Control is transferred to the operator. Accordingly, after the priority gesture 11 of the first operator, the mission vehicle 20 may be operated by the gesture 10 of the first operator.

As described above, when the control right acquisition signal is generated in several places, the control right of the mission vehicle 20 may be transferred. On the other hand, when a control right acquisition signal with a high set priority is received, the control right is transferred to an operator with a high setting.

The forward gesture 12 , the backward gesture 12 , and the stop gesture 14 are motions for applying a forward, backward, and stop signal of the mission vehicle 20 according to a set gesture, respectively. For example, the battery gesture is set in a form in which the thumb is raised upward in a clenched fist state (hereinafter referred to as a 'forward motion'), and when the operator takes a forward motion, the gesture recognition unit 110 moves forward. motion is recognized.

In addition, the backward gesture 12 is set in a form in which the thumb is lowered in a downward direction (hereinafter, referred to as 'reverse motion') in a state in which a fist is clenched, and when the operator takes a backward motion, the gesture recognition unit 110 is Reverse motion is recognized.

In addition, the stop gesture 14 is set in the form of clenching a fist (hereinafter, referred to as a 'stop motion'), and when the operator takes a stop motion, the gesture recognition unit 110 recognizes the stop motion.

Also, the steering gesture 15 is a motion for applying a signal of a left turn angle and a right turn angle when turning left, right, and left of the mission vehicle 20 according to a set gesture, respectively.

For example, the left turn gesture 15a is set in a form in which only the thumb and index finger are spread out after pointing the palm toward the mission vehicle 20 and the remaining three fingers are closed (hereinafter referred to as 'left turn motion'), so that the operator turns left When a motion is taken, the gesture recognition unit 110 recognizes a left rotation motion.

In addition, when the motion extending from the fist to the three fingers one by one (hereinafter referred to as 'left rotation angle motion') is performed together with the hand performing the left rotation motion or subsequently, using the other hand, 11 o'clock, 10 o'clock, and 9 o'clock The left turn angle signal may be applied in the direction, the 8 o'clock direction and the 7 o'clock direction, respectively. For example, after performing a left rotation motion with the left hand, if a left rotation angle motion with three fingers, ring finger, and middle finger open with the left or right hand is continuously applied, this may be applied as a signal to turn left in the 9 o'clock direction. Also, for example, together with the hand performing the left rotation motion, or subsequently, using the other hand, the motion extending from the fist to the three fingers one by one (hereinafter referred to as the 'left rotation angle motion') occurs in the 11 o'clock and 10 o'clock positions , 9 o'clock, 8 o'clock, and 7 o'clock, respectively, a left rotation angle may be applied. For example, if a left rotation motion is executed with the left hand and a left rotation angle motion with the three fingers, ring finger, and middle finger open with the right hand is applied at the same time, this may be applied as a signal to turn left in the 9 o'clock direction.

In addition, the right turn gesture 15b is set in a form in which the palm is turned toward the mission vehicle 20 side, the thumb, index finger, and three fingers are spread out and the other two fingers are closed (hereinafter referred to as 'right turn motion'), so that the operator turns right When a motion is taken, the gesture recognition unit 110 recognizes a right turn motion.

In addition, when the motion extending from the fist to the three fingers one by one (hereinafter referred to as the 'right turn angle motion') is performed together with the hand performing the right turn motion or subsequently, using the other hand, 1 o'clock, 2 o'clock, and 3 o'clock A right turn angle signal may be applied in the direction, the 4 o'clock direction and the 5 o'clock direction, respectively. For example, after performing a right turn motion with the left hand, if a right turn angle motion of opening three, ring and middle fingers is continuously applied with the left or right hand, this may be applied as a signal to turn right in the 3 o'clock direction. Also, for example, together with the hand performing the right turn motion, or in succession with the other hand, the motion extending from the fist to the three fingers one by one (hereinafter referred to as the 'right turn angle motion') occurs in the 1 o'clock and 2 o'clock positions , 3 o'clock, 4 o'clock, 5 o'clock, respectively, the right rotation angle may be applied. For example, if a right turn motion is executed with the left hand and a right turn angle motion with the three fingers, ring finger, and middle finger open with the right hand is applied at the same time, this may be applied as a signal to turn left in the 3 o'clock direction.

Also, the end gesture 16 is a motion for applying a gesture operation signal according to the end of driving of the mission vehicle 20 according to a set gesture.

For example, the end gesture 16 is set in a form in which all fingers are spread out (hereinafter, referred to as 'end motion'), and when the operator takes the end motion, the gesture recognition unit 110 recognizes the end motion. .

In addition to the end gesture 16 , a control right release gesture 17 for releasing the control right may also be applied. The control release gesture 17 may be, for example, a form in which the operator's fingers are all extended and a fist clenching action, or alternatively, in a form in which both hands are spread out.

When the control right release gesture 17 is applied to the gesture recognition unit 110, the control unit 120 receives the control right release gesture 17 signal from the gesture recognition unit 110 and transfers the control right to the operator of the next rank It can be set to hand over or return to previous control.

In addition, in one embodiment of the present invention, when changing the operating method of the mission vehicle 20, the operation mode change gesture 18 for changing may be further included.

That is, the operator may use the priority gesture 11 to obtain control right for operation of the mission vehicle 20 as well as implement an operation mode change, but in order to implement this separately, the operator operates the mission vehicle 20 For this, the priority gesture 11 for acquiring control and the operation mode change gesture 18 for changing the operation mode can be set, respectively.

5 is a flowchart schematically illustrating a vehicle control method using a gesture according to an embodiment of the present invention. 6 is a control flowchart schematically illustrating a vehicle control method using a gesture according to an embodiment of the present invention.

5 and 6, the vehicle control method using a gesture according to an embodiment of the present invention includes a control right acquisition step (S10), a gesture pattern recognition step (S20), a vehicle control command inquiry step (S30), and a task It may include a step (S40) of controlling the operation of the vehicle (20).

As described above with reference to FIGS. 1 to 4 , the gesture recognition unit 110 can be divided into a case of the vision camera sensor 111 and a case of the motion communication unit 112 . can

First, a vehicle control method using a gesture in the case of the vision camera sensor 111 may be described.

Before acquiring the control right of the mission vehicle 20 , the mission vehicle 20 may be driven through normal operation, for example, manually or by a remote control device or autonomous driving, or may be in a state of waiting for operation mode selection (mission vehicle 20 ). ) normal driving step (S110)).

In the normal driving state of the mission vehicle 20, the control unit 120 determines whether or not the priority gesture 11 is applied to the gesture recognition unit 110 (control acquisition determination step S120).

When the operator executes the priority gesture 11 (or the operation mode change gesture 18) with the mission vehicle 20, the gesture recognition unit 110 displays the gesture 10 of the operator who performed the priority gesture 11 , and a corresponding signal is applied to the control unit 120 .

The mission vehicle 20 stops the current operating state, for example, an operating state during manual driving, remote driving, or autonomous driving, and the operator can obtain control of the mission vehicle 20 through the priority gesture 11 . Yes (control acquisition step (S10)). When the operator acquires the control right, the mission vehicle may be operated in a gesture mode according to the operator's gesture (S130).

When the mission vehicle 20 is operated in the gesture mode while the operator has obtained the control right, the gesture recognition unit 110 recognizes the operator's gesture 10 pattern (gesture pattern recognition step S20). ). For example, when the operator executes a forward gesture 12, a backward gesture 12, a stop gesture 14, a steering gesture 15, and an end gesture 16, the gesture recognition unit 110 may The operator's gesture 10 pattern is recognized and a signal corresponding thereto is applied to the controller 120 .

The control unit 120 inquires for a vehicle control command corresponding to the signal received from the gesture recognition unit 110 (vehicle control command inquiry (S30)), and operates the mission vehicle 20 according to the vehicle control command. is controlled (operation control step (S40) of the mission vehicle 20).

Through the above steps, the operation of the mission vehicle 20 may be controlled. In this state, when the operator executes the control right release gesture 17 , the gesture mode of the mission vehicle 20 may be released, and the set operation method or the original operation method may be changed.

During operation of the gesture mode of the mission vehicle 20 , one operator may continuously drive the operation, but when there are a plurality of operators, the operator may be changed. Accordingly, the gesture recognition unit 110 determines whether the signal of the priority gesture 11 is applied while the mission vehicle 20 is operated in the gesture mode (priority gesture determination step S140).

While operating in the gesture mode of the mission vehicle 20 , the execution of the priority gesture 11 by another operator may occur. According to the generation of the priority gesture 11 of the other operator (priority gesture determination step S140 ), the priority gesture 11 may be detected by the gesture recognition unit 110 . The control unit 120 determines the priority according to the signal application of the priority gesture 11, transfers the control right according to the set priority, and the mission vehicle 20 is operated in the gesture mode by the changed operator. It can be (operational operator change operation step (S150)).

Next, a vehicle control method using a gesture in the case of the motion communication unit 112 may be described. In the case of the motion communication unit 112 , the operator wears a motion recognition glove 10a capable of wireless communication with the motion communication unit 112 . Accordingly, the priority right of control for the operation of the mission vehicle may be transferred to the operator wearing the motion recognition gloves 10a.

As with the vision recognition sensor described above, before acquiring the control right of the mission vehicle 20, the mission vehicle 20 is driven through general operation, for example, manual or remote control device or autonomous driving, or operation mode selection is performed. It may be in a waiting state (the general driving step of the mission vehicle 20 ( S110 )).

In the normal driving state of the mission vehicle 20, when the motion communication unit 112 communicates with the motion recognition glove 10a worn by the operator, the operation is changed to the gesture mode without a separate priority gesture or operation mode change gesture. It can be (control acquisition determination step (S120)).

The operator wearing the motion recognition gloves () through the control acquisition determination step (S120) obtains the control right (control acquisition step (S10)).

The mission vehicle 20 stops the current operating state, for example, during manual driving, remote driving, or autonomous driving, and the operating method of the mission vehicle 20 is changed to operation using a gesture, so that the gesture mode can be operated (gesture mode operation step (S130)).

When the mission vehicle 20 is operated in the gesture mode while the operator has obtained the control right, the gesture recognition unit 110 recognizes the operator's gesture 10 pattern (gesture pattern recognition step S20). ). For example, when the operator executes a forward gesture 12, a backward gesture 12, a stop gesture 14, a steering gesture 15, and an end gesture 16, the gesture recognition unit 110 may The operator's gesture 10 pattern is recognized and a signal corresponding thereto is applied to the controller 120 .

The control unit 120 inquires for a vehicle control command corresponding to the signal received from the gesture recognition unit 110 (vehicle control command inquiry (S30)), and operates the mission vehicle 20 according to the vehicle control command. is controlled (operation control step (S40) of the mission vehicle 20).

Through the above steps, the operation of the mission vehicle 20 may be controlled. In this state, when the operator executes the control right release gesture 17 , the gesture mode of the mission vehicle 20 may be released, and the set operation method or the original operation method may be changed.

During operation of the gesture mode of the mission vehicle 20 , one operator may continuously drive the operation, but when there are a plurality of operators, the operator may be changed. Accordingly, the gesture recognition unit 110 determines whether the signal of the priority gesture 11 is applied while the mission vehicle 20 is operated in the gesture mode (priority gesture determination step S140).

While operating in the gesture mode of the mission vehicle 20 , another operator may wear another motion recognition glove 10a or execute the priority gesture 11 while wearing it. The motion communication unit 112 may detect a signal of the motion recognition glove 10a of another operator, and transfer the control right to an operator of a set priority or a lower priority accordingly. In the motion communication unit 112, the mission vehicle 20 may be operated in a gesture mode by a variable operator (operator change operation step S150).

Although embodiments of the present invention have been described with reference to the above and the accompanying drawings, those of ordinary skill in the art to which the present invention pertains can practice the present invention in other specific forms without changing its technical spirit or essential features. You will understand that there is Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

10: Operator's gesture
10a: motion recognition gloves
11: Priority gesture
12: Forward gesture
13: backward gesture
14: stop gesture
15: Steering Gesture
16: end gesture
17: release control gesture
18: operation mode change gesture
100: vehicle control system using gestures
110: gesture recognition unit
111: vision camera sensor
112: motion communication unit
120: control unit

Claims (6)

a gesture recognition unit provided in the mission vehicle to detect an operator's gesture; and
a control unit for controlling driving of the mission vehicle according to the gesture of the operator detected by the gesture recognition unit;
The gesture recognition unit is provided with a priority gesture for a specific operator among a plurality of operators to obtain a control right to drive the mission vehicle,
A vehicle control system using gestures. The method of claim 1,
The gesture recognition unit operates in at least one of a vision recognition mode for detecting the operator's gesture through a vision camera sensor, and a motion communication mode for detecting the motion of the motion recognition glove mounted on the operator.
A vehicle control system using gestures. The method of claim 1,
The gesture recognition unit includes a forward gesture, a backward gesture, a stop gesture, a steering gesture, and an end gesture for terminating operation of the gesture together with the priority gesture for acquiring control rights,
A vehicle control system using gestures. obtaining, by an operator, control of the mission vehicle;
recognizing a pattern of a gesture of the operator through a gesture recognition unit of the mission vehicle;
inquiring a vehicle control command corresponding to the gesture pattern; and
Including the step of controlling the operation of the mission vehicle according to the vehicle control command,
A vehicle control method using gestures. 5. The method of claim 4,
The gesture recognition unit detects the gesture of the operator through the vision camera sensor, or detects the motion of the motion recognition glove mounted on the operator,
A vehicle control method using gestures. The method of claim 1,
When the operator is plural, the control right acquisition is determined according to a priority gesture for acquiring the priority control right of driving the mission vehicle;
A vehicle control method using gestures.

Images