KR102252852B1 - Gesture data collection and management system - Google Patents

Abstract

A gesture data collection and management system in accordance with an embodiment of the present invention comprises: a data collection unit; a control unit; and a monitoring unit. The data collection unit collects photographing data for a gesture motion performed by a user and sensor data obtained by sensing a body motion and provides the collected photographing data and the sensor data to the control unit. The control unit performs a labeling operation of automatically assigning a characteristic value to the gesture data obtained from the data collection unit, and performs control so that the photographing data and the sensor data are recorded with a timestamp. The monitor unit displays performance information of data collection and management operation, received through the control unit. The data collection unit may include a sensor data collection unit which is worn on the user's body and senses the user's movement, and a metadata collection unit which senses metadata to assist the sensor data. The gesture data collection and management system can reduce the time required for checking and correcting data errors.

Description

Gesture data collection and management system

The present invention relates to a system for collecting and managing gesture data from a wearable gesture collection device in order to distinguish and recognize a user's gesture.

In performing gesture recognition, collecting accurate gesture data is an important process that affects recognition accuracy. Gesture data can be collected using a variety of devices such as cameras, EMG, IMU sensors, and Kinect. In this way, user data collected through various devices needs to be modified or cut out before being used in an experiment. In the case of recognizing a gesture using supervised learning, a labeling process is required to distinguish temporal boundaries such as a start point or an end point of the gesture, and to display the correct answer in the divided section. There are ways to use a toggle switch to label gesture data, or to record with an observer, but you can't see the gesture part afterwards. Therefore, for accurate labeling, the experimenter must directly label, and an intuitive clue for labeling work is required.

However, conventionally, a method of automatically performing labeling of such gesture data has not been proposed, and thus, there has been a lot of inconvenience in performing a task using gesture data.

The present invention was devised to propose an IMU sensor data collection and management system using a camera and a wireless sensor device. Specifically, in the present invention, when collecting gesture data, IMU sensor data and an image are recorded together with a timestamp, so that intuitive information can be obtained, and the experimenter finds a clue of labeling through the stored image and uses the timestamp to record the image. The purpose is to be able to search IMU sensor data corresponding to.

In order to achieve the above object, the gesture data collection and management system according to an embodiment of the present invention may include a data collection unit, a control unit, and a monitor unit, and the data collection unit The photographing data and sensor data sensing body motion are collected and provided to the control unit, and the control unit performs a labeling operation of automatically assigning a characteristic value to the gesture data obtained from the data collection unit, including photographing data and sensor data. Is controlled to be recorded with a timestamp, the monitor unit displays information on the performance of data collection and management operations received through the control unit, and the data collection unit collects sensor data that is worn on the user's body and senses the movement of the user It may be configured to include a meta data collection unit that senses meta data to assist sub and sensor data.

In the present invention, when collecting gesture data, IMU sensor data and an image are recorded together with a timestamp. Accordingly, the present invention provides intuitive information through a graph that visualizes the physical information of the IMU sensor data in the data collection program and the camera image, so that the experimenter finds the clues of labeling through the stored image and uses a timestamp to correspond to the image. IMU sensor data can be easily retrieved.

In addition, according to the present invention, when collecting data, the experiment participant can check the graph and image of the collected data in real time through the push button and the monitor screen, and since the gesture data can be saved with an automated collection program, data errors can be checked. And reduce the time to make corrections.

1 is a diagram illustrating a configuration of a gesture data collection and management system according to an embodiment of the present invention.
2 is a diagram showing a configuration of a control unit according to an embodiment of the present invention.
3 is a diagram illustrating a position setting value between a camera and a user for gesture input according to an embodiment of the present invention.
4 is a flowchart illustrating a sequence of a gesture collection operation according to an embodiment of the present invention.
5 to 7 are diagrams illustrating an example of a gesture management operation according to an embodiment of the present invention.

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in a variety of different forms. It is provided to fully inform the technician of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terms used in the present specification are for describing exemplary embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase. As used herein, “comprises” and/or “comprising” do not exclude the presence or addition of one or more other elements other than the mentioned elements. Throughout the specification, the same reference numerals refer to the same elements, and "and/or" includes each and all combinations of one or more of the mentioned elements. Although "first", "second", and the like are used to describe various elements, it goes without saying that these elements are not limited by these terms. These terms are only used to distinguish one component from another component. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical idea of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used with meanings that can be commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not interpreted ideally or excessively unless explicitly defined specifically.

The term "unit" or "module" used in the specification refers to a hardware component such as software, FPGA or ASIC, and the "unit" or "module" performs certain roles. However, "unit" or "module" is not meant to be limited to software or hardware. The "unit" or "module" may be configured to be in an addressable storage medium or may be configured to reproduce one or more processors. Thus, as an example, "sub" or "module" refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, Includes procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays and variables. Components and functions provided within "sub" or "modules" may be combined into a smaller number of components and "sub" or "modules" or into additional components and "sub" or "modules". Can be further separated.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe the correlation between a component and other components. Spatially relative terms should be understood as terms including different directions of components during use or operation in addition to the directions shown in the drawings. For example, if a component shown in a drawing is turned over, a component described as "below" or "beneath" of another component will be placed "above" the other component. I can. Accordingly, the exemplary term “below” may include both directions below and above. Components may be oriented in other directions, and thus spatially relative terms may be interpreted according to the orientation.

In the present specification, a computer refers to all kinds of hardware devices including at least one processor, and may be understood as encompassing a software configuration operating in a corresponding hardware device according to an embodiment. For example, the computer may be understood as including all of a smartphone, a tablet PC, a desktop, a laptop, and a user client and application running on each device, and is not limited thereto.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating a configuration of a gesture data collection and management system according to an embodiment of the present invention.

As shown in FIG. 1, a gesture data collection and management system according to an embodiment of the present invention may include a data control unit 100, a data collection unit 110, and a monitor unit 120. In this case, the data collection unit 110 may include a meta data collection unit 111 and a sensor data collection unit 112.

Although not shown in FIG. 1, the system may further include a storage unit and a communication unit, and each component may be connected to the control unit 100 to operate.

Meta data is auxiliary input data stored together with sensor data (user motion detection data sensed by an IMU sensor), and may mean image data photographed through a camera, button input data input through a push button, and the like. The meta data collection unit 111 is a device that collects such meta data, and may include, for example, a push button or a camera. The push button may be designed to be connected to a long line in order to sufficiently secure a range of activities of the experiment participants. In addition, in the case of the camera, according to an embodiment, it may be installed in a position where the upper body of an experiment participant is all visible.

The sensor data collection unit 112 may include an inertial measurement unit (IMU) sensor. The IMU sensor can be wirelessly connected in order not to limit the behavior of the participant in the experiment. That is, the sensor data collection unit 112 may perform wireless data transmission/reception with the control unit 100.

According to an embodiment, the IMU sensor may be a device assembled with a smart phone, a myo armband (a motion detection sensor for arm wear), an Arduino and MPU-6050 6-axis gyro sensor, and a Bluetooth module as an input device. In addition, according to an embodiment, the IMU sensor may be set to be sampled at an interval of 50 ms.

In addition, the control unit 100 may obtain metadata and sensor data from the data collection unit 110 (meta data collection unit 111 and sensor data collection unit 112), and may classify and manage the acquired data. In addition, the control unit 100 may randomly select specific gesture data to be acquired and output it to the monitor unit 120, and collect gesture data of a user corresponding thereto through the data collection unit 110. According to various embodiments, when a user inputs gesture data through a data collection unit 110 such as a camera or an IMU sensor, the control unit 100 directly transmits required setting value information to the data collection unit 110 to receive a sensor setting value. You can ask to change it.

In addition, the control unit 100 may provide and display various information to the monitor unit 120. For example, the controller 100 may transmit and output a camera image captured by the monitor unit 120, gesture guide information to be input by a user, and acquired gesture data (graph information).

In order to describe in more detail with respect to the control unit 100, reference will be made to FIG. 2.

2 is a diagram showing the configuration of a control unit according to an embodiment of the present invention.

As shown in FIG. 2, the control unit 100 according to an embodiment of the present invention may include a sensor setting unit 101, a gesture selection unit 102, a data management unit 103, an output data matching unit 104, and a storage unit 105.

The sensor setting unit 101 of the control unit 100 may check and adjust various set values of a sensor (or data collection unit) required to collect gesture data of a user. For example, the sensor setting unit 101 may adjust setting values, such as photographing brightness and the number of frames per second, of a camera device corresponding to the meta data collection unit 111.

The sensor setting unit 101 may request to determine an appropriate position of the camera by calculating a set value for setting an appropriate camera position and outputting the same to the monitor unit 120. For example, the proper position of the camera may be a position such that the upper body of the experiment participant (the user who will input a gesture) is all visible. That is, the sensor setting unit 101 can detect images such as the user's (experimental participant) face, the user's shoulder line, and the user's arm input through the camera before the gesture data collection operation is started. It is possible to judge whether the entire upper body of the (experimental participant) is visible. When it is determined that the upper body of the user (experimental participant) is confirmed, the sensor setting unit 101 determines that the corresponding position is in an appropriate position, and when it is determined that the corresponding position is an appropriate position, a sign that approves gesture shooting. May be transmitted to the monitor unit 120 and displayed.

The sensor setting unit 101 may also determine whether not only the location of the camera, but also a photographing condition of the camera is suitable for photographing a gesture. For example, the sensor setting unit 101 may determine whether or not elements such as illuminance, resolution, and focus are set to be inappropriate for an image photographed through the camera, and provide an adjustment request message to the monitor 120.

In the case of an image photographed by a camera, the brightness of the screen is dark or the focus is set so that the gesture of the user (experiment participant) may not be specified. Therefore, it is essential to detect whether such a condition is properly set before the gesture is performed, and to correct the set value in advance. Therefore, the sensor setting unit 101 can determine in advance the necessity of adjusting the image setting values (illuminance, resolution, focus, etc.) of the camera, and when the necessity of adjustment is determined, the monitor unit 120 controls to output an adjustment request message. can do.

According to various embodiments, the sensor setting unit 101 may set information such as the number of frames and sampling intervals of a camera image. For example, the sensor setting unit 101 samples a camera image at 5 frames/s and may adjust a sampling interval. In this case, the sensor setting unit 101 may adjust the sampling interval differently according to various conditions. For example, the sensor setting unit 101 may adjust the image sampling interval based on the brightness of the captured image and the expected distance between the user (experimental participant) and the camera.

As shown in FIG. 3, the sensor setting unit 101 presets information such as the distance between the camera and the user, and the size of the area in which the user draws text with a hand in the air, and adjusts the user's position to an appropriate position. It is possible to perform a guide operation for the purpose.

3 is a diagram illustrating a position setting value between a camera and a user for gesture input according to an embodiment of the present invention.

The position setting value between the camera and the user set by the sensor setting unit 101 may be designated such that the distance between the user and the camera becomes a specific value (eg, 1 m). In addition, the sensor setting unit 101 uses a specific point of the user's body (eg, the center of the shoulder line) as the center point, and the horizontal and vertical lengths are designated as specific values (eg, 1.2m). Can be set to. Alternatively, the sensor setting unit 101 may set the central portion of the aerial area to be at a certain height (eg, 130 ts) from the floor. Accordingly, when it is determined that the position setting value between the camera and the user is not satisfied, the sensor setting unit 101 may request the user to adjust the distance between the camera and the user or the height of the camera.

According to various embodiments, the aerial area, which is an area in which a user inputs a gesture, may be visually expressed through a structure, and in this case, the structure may be implemented in a form in which 9 cells are divided. Accordingly, the sensor setting unit 101 may determine whether or not it is ready to input a gesture by first checking the location of the aerial area through the structure and first checking the size of the structure and the distance from the camera. have. In addition, when it is determined that the gesture input condition is satisfied based on the size of the structure and the separation distance between the structure and the camera, the sensor setting unit 101 may display an approval signal notifying this on the screen.

In addition, the sensor setting unit 101 may designate a set value related to a push button and request correction thereof. For reference, the push button may be a device designed to distinguish and receive various types of input values. For example, the push button may be implemented to recognize one-time input (push button push), two-time input (double-press), and long-time input (long-press). Accordingly, the sensor setting unit 101 can set a corresponding setting value (eg, gesture input confirmation) when the push button is input once and a corresponding setting value (eg, gesture input cancellation) when the push button is input twice. Yes, and you can change the setting value.

In addition, the sensor setting unit 101 may designate and control a set value related to the sensor data collection unit 112 as well as the meta data collection unit 111.

The sensor setting unit 101 may designate or change a setting value for a sampling interval (eg, a default value of 50 ms) of the sensor data collection unit 112 (eg, an IMU sensor) and a sensing sensitivity of the IMU sensor.

According to various embodiments, the sensor setting unit 101 may determine the age range or arm length of the user (experimental participant) based on the body size of the user (experimental participant) photographed through the camera, and based on this (or Based on the user's body size or age information directly input by the user), predict the gesture size expected to be suitable for the user (experimental participant), and accordingly, the sensing sensitivity of the sensor data collection unit (IMU sensor) You can ask to make adjustments. For example, in order to prevent the problem that the sensing sensitivity of the sensor data collection unit (e.g., IMU sensor) is set for infants and is detected as a meaningful gesture with relatively little movement, the user (experimental participant) confirmed through the camera The appropriate sensing sensitivity corresponding to the body size (height, shoulder width, arm length, etc.) can be calculated and applied to the sensor data collection unit.

The gesture selection unit 102 may select an arbitrary gesture to be collected and output the selected gesture to the monitor unit 120. Accordingly, the gesture selection unit 102 may support collection of a gesture corresponding to data output to the monitor unit 120 from a user (experiment participant).

At this time, gestures that can be collected may include text data written in the air, sign language, and other various hand and body movements.

According to an embodiment, the gesture selection unit 102 may sequentially select items to be collected from among the meanings of gestures previously stored in a storage unit (not shown) and display them on the monitor unit 120. For example, the meaning items of gestures previously stored in the storage unit include numeric items (e.g., 1, 2, 3, 4, etc.), letter items (e.g., A, B, C, D, etc.), sign language items, etc. It can be configured. Accordingly, when a number item is selected as a gesture meaning item to be collected, the gesture selector 102 may display a numerical value increasing from 0 to 1 on the screen sequentially in ascending order. In this way, the gesture selection unit 102 may control the monitor unit 120 to display an arbitrary number on the screen, and request a user (experiment participant) to perform a gesture having a meaning displayed on the screen.

In this case, the gesture selection unit 102 may change and select a gesture meaning item in response to a response of a user (experiment participant). After performing the gesture inputting operation once, the user can take an action indicating that the gesture inputting operation has been completed, and at this time, the gesture selection unit 102 can determine the user's (experimental participant) action. have. According to an embodiment of the present invention, a user (experiment participant) may notify whether he or she has completed a gesture input operation by inputting a push button.

In addition, when it is determined that the time point at which the user's motion is not detected exceeds a preset time, the gesture selector 102 may perform an operation such as changing an item of a gesture requesting an input.

The gesture management unit 103 may perform an operation of managing gesture data acquired through the data collection unit 110.

Specifically, after the gesture selection unit 102 requests the user to input a gesture, the gesture management unit 103 may match and store the corresponding gesture with a previously displayed semantic item when a gesture is input by the user.

Specifically, as illustrated in FIG. 4, the gesture manager 103 may perform a gesture collection operation.

4 is a flowchart illustrating a sequence of a gesture collection operation according to an embodiment of the present invention.

As illustrated in FIG. 4, the gesture management unit 103 may display a setting menu on the screen, and at this time, when the user inputs a push button, the gesture management unit 103 may control to perform an operation to guide a gesture and direction to be collected. . In response, the gesture selection unit 102 may select a gesture item (eg, a gesture for drawing a number 1) to be collected from among pre-stored gesture items (eg, number items such as 1, 2, 3) and display it. have. For example, when a gesture for the number 1 is to be collected, the gesture selection unit 102 may control to display the text for the number '1' on the monitor unit 120. In addition, the gesture management unit 103 may start a data collection operation together with a gesture and direction guidance (such as a gesture input direction).

Thereafter, the gesture management unit 103 may perform an operation of acquiring gesture data and storing the corresponding data as the user's push button input is received. In addition, a graph related to the gesture data can be displayed. In that state, when a user's push button is input, the gesture management unit 103 may control to play a camera image of the collected data.

On the other hand, when a second input (eg, double-pressing) to the push button is performed in the data storage operation step, an operation (gesture re-collection operation) for re-collection of the target gesture may be performed. In addition, when a third input to the push button (e.g., long pressing of the push button) is performed in the data storage operation step, the gesture management unit 103 controls the gesture selection unit 102 to change the item of the gesture to be collected to the next item. Accordingly, the gesture selection unit 102 may select and display the next gesture item.

When storing the acquired gesture data, the gesture management unit 103 may store items such as an experiment participant ID, a gesture number, a gesture position, an execution time, a sensor wearing position, a sampling interval, and the like in a file name. For example, the gesture data may be stored as a CSV file, and the stored data may include a time index, an acceleration 3 axis, a gyro 3 axis, and a button activation tag. The button activation tag may be recorded to provide additional information to the experimenter's labeling operation. In addition, the gesture management unit 103 may classify and store a section corresponding to a gesture in a gesture according to whether or not a gesture has been performed, and a data section in which a gesture has been taken can be classified as 1 and a section in which a gesture is not taken can be classified as 0.

The gesture data acquisition and storage operation performed by the gesture management unit 103 will be described in more detail with reference to FIGS. 5 to 7.

5 to 7 are diagrams illustrating an example of a gesture management operation according to an embodiment of the present invention.

First, FIGS. 5 and 6 show screens of a gesture data collection program and a gesture data management program.

As shown in FIG. 5, the gesture management unit 103 according to an embodiment of the present invention may support an operation of a gesture collection program. At this time, when the gesture management unit 103 acquires the user's gesture through an IMU sensor worn on the user's body or a camera positioned in front of the user through the collection program, the signal data corresponding to the gesture is collected as it is. And it can be processed. In this case, the signal data may include values such as a gyro sensing value and an acceleration sensing value, a position of a graph peak, an interval between the peaks, and the like.

According to an embodiment, the gesture management unit 103 may support displaying a gesture graph, a camera image, and a gesture indicator through a gesture collection program. The gesture management unit 103 supports a user to check a gesture collection situation in real time through a camera image displayed on a screen. In addition, the gesture management unit 103 may display a gesture indicator to inform a user (experiment participant) of a gesture to be collected and a direction to be directed to the gesture by subdividing it. The gesture indicator may be a device that divides nine cells on a screen and provides meaning information of a gesture in a specific cell. In addition, the gesture management unit 103 may notify the user of whether gesture data is collected by changing the background color of the gesture indicator.

In addition, the gesture management unit 103 may support the user to directly select a meaning of a gesture to be collected and then perform a gesture operation corresponding to the meaning. To this end, the gesture management unit 103 allows the user to directly input numbers or characters in the collection program, and then, when it is determined that the preparation of the data collection unit such as an IMU sensor or a camera is completed, may collect the user's gesture motion.

In addition, FIG. 6 shows a data management program, and the gesture management unit 103 may perform a labeling operation on the collected gesture. The collected gesture data may be a combination of camera data and sensor data. In addition, the combined data is recorded together with time information, so that the sensor data generated at the same time as the camera image information can be checked. Accordingly, when labeling the collected sensor data, the gesture management unit 103 provides gesture input characteristic information detected from camera information captured at the same time point (e.g., gesture input start position, user body size information (children, adults, etc.)). Can be used as labeling information.

In addition, the gesture management unit 103 may determine a starting point and an ending point of the collected gesture data based on a graph as shown in FIG. 7 through a data management program. Through this, the gesture management unit 103 may extract the execution time of the specific gesture, and data values near the starting point and the ending point of the specific gesture.

The output data matching unit 104 may perform an operation of matching a value of data to be output in response to a data collection operation. For example, the output data matching unit 104 may determine an input size of a gesture and control to set a display size of output data corresponding to the corresponding size.

According to various embodiments, the output data matching unit 104 determines whether the collected data satisfies a reference condition, and when the reference condition is not satisfied, informs the user (experiment participant) that collection of previously performed gestures has been invalidated. You can ask to re-enter the gesture. In this case, the output data matching unit 104 may match a notification sound indicating invalidation of a previously performed gesture.

In this case, the output data matching unit 104 may match and output different notification sounds in response to a reason for invalidating the performed gesture. For example, when it is determined that the gesture motion size exceeds the reference area (e.g., when a user's motion is detected outside the shooting area of the camera), the output data matching unit 104 matches the first notification sound to match the user's motion range. You can ask to reduce it.

Similarly, when the output data matching unit 104 determines that the size of the gesture performed by the user is performed within a range below the reference area (e.g., when the IMU sensor detects but the camera does not detect the user's movement), The output data matching unit 104 may request to expand the motion range of the user by matching the second notification sound.

And the output data matching unit 104 based on the labeling information (e.g., experiment participant ID, gesture number, gesture position, trial time, sensor wearing position, sampling interval, etc.) given to specific gesture data by the data management unit 103, It is possible to control to match requests related to the distraction phenomenon of collected data and output them. In order to increase the gesture recognition rate, a process of collecting various types of data is essential. Even gestures that have the same meaning (e.g., a gesture that draws the number 1 in the air) have different gesture input characteristics such as the speed at which the movement is performed, the location of the starting point of the movement, the length of the gesture, and the position of the user wearing the sensor. can do. However, if all of the above-mentioned conditions are collected only similar data within the reference range, it may become an obstacle to the improvement of the gesture recognition rate in the future, and thus efforts to collect various types of gestures may be required.

Accordingly, when the output data matching unit 104 determines that labeling information of previously collected gesture data or classification information of other gesture input characteristics (eg, the position of the motion start point, the motion length of the gesture, etc.) is equal to or greater than the reference number of times. , A user (experiment participant) may be asked to input a different type of gesture. According to an embodiment, the experiment according to an embodiment of the present invention may propose to repeat the operation of writing the same number in the air to one user 10 or more times. Accordingly, the user can maintain the same gesture input characteristics, such as drawing a number less than the reference size (eg, 10cm*10cm) without even knowing it. In order to make the user aware of such a problem, the output data matching unit 104 suggests changing the gesture input characteristic when it is determined that gestures having the same gesture input characteristic more than a reference number of times (e.g., 5 times) have been collected. Can be controlled to match and output it.

For example, the output data matching unit 104 determines that the cumulative number of times a user has performed a gesture operation below a reference size (e.g., 10cm*10cm area) is greater than or equal to the reference number (e.g., 5 times), When it is determined that the cumulative number of gesture actions performed by a user within a reference time (e.g., 1 second) or less is determined to be greater than or equal to the reference number (e.g., 5 times), and the gesture to be performed later You can ask to increase the implementation time.

In short, the gesture data collection and management system according to an embodiment of the present invention may include a data collection unit, a control unit, and a monitor unit, and the data collection unit senses photographing data and body movements for gesture actions performed by a user. One sensor data is collected and provided to the control unit, and the control unit performs a labeling operation of automatically assigning a characteristic value to the gesture data obtained from the data collection unit, so that the photographed data and the sensor data are recorded together with a timestamp. Control, and the monitor unit displays information on the performance of data collection and management operations received through the control unit, and the data collection unit is worn on the user's body to sense the user's movement, and the data collection unit assists the sensor data. It may be configured to include a meta data collection unit that senses the meta data for.

In this case, the sensor data collection unit includes an IMU (Inertial Measurement Unit) sensor worn on a part of the user's body, and the meta data collection unit includes a push button for inputting information on whether a gesture operation has been completed, and a front surface of the user. It may be configured to include a camera that is provided in the user's gesture performance image.

In addition, the control unit checks and adjusts the set value of the data collection unit required for collecting gesture data, but determines whether the user's upper body is detected in the captured image, the distance between the user and the camera based on the camera image. It may include a sensor setting unit that determines whether or not the camera is placed in an appropriate position, and, when it is determined that the camera is installed at an appropriate position, controls the monitor unit to display a sign for approving photographing.

In addition, the sensor setting unit may predict the user's suitable gesture size in response to the user's body size and age information, and adjust the gesture sensing sensitivity of the sensor data collection unit in response to the predicted suitable gesture size.

In addition, the control unit sequentially selects semantic items of gestures to be collected and controls them to be displayed on the monitor unit, and includes a gesture selection unit that controls the item of gestures to be collected to be changed to the next item as the input of the push button is detected. can do.

When storing the acquired gesture data, the control unit stores the gesture input characteristic information including the experiment participant ID, the gesture number, the gesture position, the trial time, the sensor wearing position, and the sampling interval in the file name, and stores the performed gesture. Is labeled with gesture input characteristic information, but records the data collected by the metadata collection unit and the sensor data collected by the sensor data collection unit together with a time stamp, so that the gesture data can be easily searched later. It may include a data management unit to manage.

In addition, when it is determined that gestures having the same gesture input characteristic more than a reference number of times have been collected, the controller may include an output data matching unit that matches and outputs a guide phrase suggesting to change the input characteristic of a gesture to be performed later to the user, The gesture input characteristic may include at least one of a gesture execution speed, a position of a starting point of a gesture operation, a length of a gesture, and a position of wearing a sensor.

Although the present invention has been described in detail with reference to the above-described examples, those skilled in the art can make modifications, changes, and modifications to the examples without departing from the scope of the present invention. In short, in order to achieve the intended effect of the present invention, it is not necessary to separately include all the functional blocks shown in the drawings or to follow all the orders shown in the drawings as shown in the order shown. Note that it may fall within the range.

100: control unit
101: sensor setting unit
102: gesture selection unit
103: data management unit
104: output data matching unit
110: data collection unit
111: metadata collection unit
112: sensor data collection unit
120: monitor unit

Claims (9)

In a gesture data collection and management system comprising a data collection unit, a control unit, and a monitor unit,
The data collection unit
Collects photographing data for gesture motions performed by the user and sensor data for sensing body motions and provides them to the controller
The control unit
Performing a labeling operation of automatically assigning characteristic values for gesture data acquired from the data collection unit, but controlling to record photographing data and sensor data together with a timestamp, and
The monitor unit
Displays information on the performance of data collection and management operations received through the control unit,
The data collection unit
Consisting of including a sensor data collection unit that is worn on the user's body to sense the movement of the user and a meta data collection unit that senses metadata to assist the sensor data,
The control unit
Check and adjust the settings of the data collection unit required to collect gesture data,
Based on the distance between the user and the camera based on the image taken by the camera, and whether the camera is in an appropriate position based on whether the user's upper body is detected in the captured image, and if it is determined that the camera is installed in the appropriate position, a sign to approve the shooting Includes; a sensor setting unit for controlling the monitor unit to display
The sensor setting unit
A gesture data collection and management system, comprising: predicting a suitable gesture size of a user in response to information on a user's body size and age, and adjusting a gesture sensing sensitivity of a sensor data collection unit in response to the predicted suitable gesture size. The method of claim 1,
The sensor data collection unit includes an IMU (Inertial Measurement Unit) sensor worn on a part of the user's body, and the metadata collection unit is provided on the front of the user and a push button for inputting information on whether the gesture operation has been completed. Gesture data collection and management system comprising a camera for capturing an image of a user's gesture performance. delete delete The method of claim 2,
The control unit
Control to sequentially select the semantic items of gestures to be collected and display them on the monitor,
And a gesture selector configured to change an item of a gesture to be collected into a next item when the push button input is detected. The method of claim 1
The control unit
When storing the acquired gesture data, the file name includes the gesture input characteristic information including the experiment participant ID, the gesture number, the gesture position, the trial time, the sensor wearing position, and the sampling interval, and stores it,
Labeling the performed gesture with gesture input characteristic information, but recording the data collected by the metadata collection unit and the sensor data collected by the sensor data collection unit together with a time stamp, making it easy to search later. Gesture data collection and management system comprising a; data management unit for managing the gesture data so as to.

delete delete delete

Images