KR20220149053A - Artificial Neural Network Device and Method for Automating Seat Position - Google Patents

Abstract

The present invention relates to an artificial neural network apparatus and method for automating a seat position of a car seat. To this end, provided are: an upper body vector generation module comprising an artificial neural network module in which a first CNN module and the first LSTM module are serially combined with time-series upper body pressure distribution data output from the upper pressure distribution sensor mounted on the upper cushion of the car seat as input data and an upper body vector as output data; a lower body vector generation module comprising an artificial neural network module in which a second CNN module and a second LSTM module are serially combined with a time-series lower body pressure distribution data output from the lower pressure distribution sensor mounted on the lower cushion of the car seat as input data and a lower body vector as output data; and a seat motion vector generation module comprising an artificial neural network module in which Softmax is continuously combined with a third LSTM module that takes a concatenated vector of the upper body vector and the lower body vector as input data and a seat motion vector as output data.

Description

Artificial Neural Network Device and Method for Automating Seat Position

The present invention relates to an artificial neural network apparatus and method for automating a seat position of a car seat.

In a car seat, especially a driver's seat car seat, if the seat position is appropriately adjusted according to the user's body type, the user can drive for a long time in a posture that does not apply excessive load to a specific body part, and secure visibility and steering wheel/ It is possible to secure the operation efficiency of the brake/accelerator, etc., thereby increasing the safety. In addition, the UX, in which the car seat automatically adjusts the seat position when the driver turns on the engine or sits down, gives a great satisfaction to the driver, so various automakers are providing seat position automation UX through seat position memory.

However, the existing seat position automation simply saves the seat position set by the user and controls the seat when calling the seat position, without considering the user's body type or the relative position with the vehicle control unit. did. Although some automakers have recently launched a smart posture control system that automates the seat position, steering column position, outside mirror angle, and HUD position to suit the user, it receives the user’s height and weight information and records the height/weight. It was just a rule-based control system that controls the seat with a set position (Hyundai Motors The New Grandeur, second-generation smart posture control system, etc.).

Republic of Korea Patent Registration 10-2153621, Control method of car seat, Hyundai Transys Co., Ltd. US Registered Patent US 10023075 B2, Automatically adjusting vehicle seat back supports of vehicle seat assemblies, Toyota Motor Engineering & Manufacturing North America, Inc US Patent US 8433482 B2, Method for automatically adjusting a headrest of a motor vehicle seat, and device for carrying out said method, Brose Fahrzeugteile GmbH & Co. KG, Coburg US 2020-0282867 A1, METHOD AND APPARATUS FOR INTELLIGENT ADJUSTMENT OF VEHICLE SEAT, VEHICLE, ELECTRONIC DEVICE, AND MEDIUM, SHANGHAI SENSETIME INTELLIGENT TECHNOLOGY CO., LTD.

It is an object of the present invention to provide an artificial neural network apparatus and method for automating the seat position of a car seat that automatically provides an optimal car seat position according to a user's body type.

Hereinafter, specific means for achieving the object of the present invention will be described.

An object of the present invention is to use time series upper body pressure distribution data output from the upper pressure distribution sensor mounted on the upper cushion of the car seat as input data, and the first CNN module and the first LSTM module using the upper body vector as output data are continuously combined Upper body vector generation module including an artificial neural network module in the form; A type of artificial neural network in which a second CNN module and a second LSTM module that use time series lower body pressure distribution data output from the lower pressure distribution sensor mounted on the lower cushion of the car seat as input data and lower body vectors as output data are continuously combined a lower body vector generation module comprising a module; and a 3rd LSTM module using a combination vector in which an upper body vector and a lower body vector are concatenated as input data and a sheet movement vector as output data, and an artificial neural network module in which Softmax is continuously combined. Including, the seat movement vector can be achieved by providing an artificial neural network device for automating the seat position, characterized in that it is configured in a dimension corresponding to the number of movable directions of the car seat.

In addition, the sheet control determination module for determining whether to proceed with the sheet control or to stop the sheet control based on the magnitude of the sheet movement vector; may be characterized in that it further comprises.

In addition, when it is determined that the sheet control is performed in the sheet control determination module, a compiler that compiles the sheet movement vector into the language of the sheet control unit may be further included.

In addition, in the learning session of the upper body vector generating module, the lower body vector generating module, and the seat movement vector generating module, the user's upper body pressure distribution in a state where both hands grip both sides of the steering wheel and the right foot rests on the brake pedal It may be characterized by using the data and lower body pressure distribution data as a reference.

Another object of the present invention is to continuously combine the first CNN module and the first LSTM module using time series upper body pressure distribution data output from the upper pressure distribution sensor mounted on the upper cushion of the car seat as input data and upper body vector as output data. An upper body vector generating step of receiving, by an upper body vector generating module including an artificial neural network module in the form of an artificial neural network module, the upper body pressure distribution data as input data and outputting the upper body vector as output data; An artificial neural network module in which a second CNN module and a second LSTM module that use time series lower body pressure distribution data output from the lower pressure distribution sensor mounted on the lower cushion of the car seat as input data and lower body vectors as output data are continuously combined a lower body vector generating step in which a lower body vector generating module comprising: receiving the lower body pressure distribution data as input data and outputting the lower body vector as output data; and a sheet motion vector generation module including an artificial neural network module in which an upper body vector and a lower body vector are concatenated (concatenated) as input data and a third LSTM module that uses a sheet motion vector as output data and Softmax are continuously combined This, a sheet movement vector generating step of receiving the combined vector as input data and outputting the sheet movement vector as output data; Including, the seat movement vector can be achieved by providing a seat position automation method using an artificial neural network, characterized in that configured in a dimension corresponding to the number of movable directions of the car seat.

As described above, according to the present invention, there are the following effects.

First, according to an embodiment of the present invention, there is an effect that can automatically provide an optimal car seat position according to the user's body type.

Second, according to an embodiment of the present invention, the effect of automatically setting the seat position according to the user's body type is generated without inputting the user's body type in advance or presetting the seat position according to the user's body type.

The following drawings attached to the present specification illustrate preferred embodiments of the present invention, and serve to further understand the technical spirit of the present invention together with the detailed description of the present invention, so the present invention is limited only to the matters described in those drawings should not be interpreted as
1 is a schematic diagram showing an artificial neural network device for automating a seat position according to an embodiment of the present invention;
2 is a schematic diagram showing a specific configuration of an artificial neural network device 1 for automating a seat position according to an embodiment of the present invention;
3 is a schematic diagram showing a specific configuration of the upper body vector generating module 2 and the lower body vector generating module 3 according to an embodiment of the present invention;
4 and 5 are exemplary views of ConvNet included in the upper body vector generating module 2 and the lower body vector generating module 3 according to an embodiment of the present invention.

Hereinafter, embodiments in which those of ordinary skill in the art can easily practice the present invention will be described in detail with reference to the accompanying drawings. However, in the detailed description of the operating principle of the preferred embodiment of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

In addition, the same reference numerals are used throughout the drawings for parts having similar functions and functions. Throughout the specification, when it is said that a specific part is connected to another part, this includes not only a case in which it is directly connected, but also a case in which it is indirectly connected with another element interposed therebetween. In addition, the inclusion of specific components does not exclude other components unless otherwise stated, but means that other components may be further included.

Artificial neural network device and method for automating seat position

1 is a schematic diagram illustrating an artificial neural network device for automating a seat position according to an embodiment of the present invention. As shown in FIG. 1 , the artificial neural network device 1 for automating the seat position uses the pressure distribution data of the pressure distribution sensor mounted on the car seat as input data, and the seat control data of the seat control unit in the vehicle ECU as output data. can be configured to

2 is a schematic diagram showing a specific configuration of an artificial neural network apparatus 1 for automating a seat position according to an embodiment of the present invention, and FIG. 3 is an upper body vector generating module 2 according to an embodiment of the present invention; It is a schematic diagram showing the specific configuration of the lower body vector generation module 3 . As shown in Figures 2 and 3, the artificial neural network device 1 for automating the seat position according to an embodiment of the present invention is the upper body pressure distribution output from the upper pressure distribution sensor 10 mounted on the upper cushion of the car seat. It may be configured to receive the data 100 and the lower body pressure distribution data 200 output from the lower pressure distribution sensor 20 mounted on the lower cushion of the car seat and output the seat control data 300 input to the seat control unit. . The artificial neural network device 1 for automating seat position according to an embodiment of the present invention includes an upper body vector generating module 2, a lower body vector generating module 3, a seat movement vector generating module 4, and a seat control determination module ( 5), a compiler 6 may be included.

The upper body vector generation module 2 uses the upper body pressure distribution data 100 output from the upper pressure distribution sensor 10 mounted on the upper cushion of the car seat as input data, and the CNN module and LSTM module using the upper body vector as output data. It may be composed of an artificial neural network module in the form of continuous coupling. In this case, the upper body pressure distribution data 100 may be input in a time series and configured to generate a time series upper body vector.

The lower body vector generation module 3 uses the lower body pressure distribution data 200 output from the lower pressure distribution sensor 20 mounted on the lower cushion of the car seat as input data, and the CNN module and LSTM module using the upper body vector as output data. It may be composed of an artificial neural network module in the form of continuous coupling. In this case, the lower body pressure distribution data 200 may be input in a time series and configured to generate a lower body vector in a time series.

The sheet motion vector generation module (4) is composed of an artificial neural network module in the form of an LSTM module that uses a joint vector concatenated with an upper body vector and a lower body vector as input data, and an LSTM module that uses a sheet motion vector as output data, and Softmax. can The seat movement vector may be configured in a dimension corresponding to the number of movable directions of the car seat.

The sheet control determination module 5 is a module that determines whether to proceed with sheet control or stop sheet control based on the magnitude of the sheet movement vector.

The compiler 6 is a module that compiles the sheet movement vector into the language of the sheet control unit when it is determined in the sheet control determination module 5 to proceed with sheet control.

Regarding the learning session of the upper body vector generating module 2, the lower body vector generating module 3, and the seat movement vector generating module 4, both hands of the user grip both sides of the steering wheel and the right foot rests on the brake pedal The resting state is defined as the reference state, and the upper body pressure distribution data (100) and lower body pressure distribution data (200) of the ground truth posture are set as the ground truth posture, which is a stable posture that enables visibility, smooth steering and pedal operation, and long-term driving. is labeled with the sheet movement vector value 0, and the sheet movement vector value labeled in the upper body pressure distribution data 100 and the lower body pressure distribution data 200 of the posture in which the seat is moved by a preset vector is set to the preset vector to compose the training data. In the learning session of the upper body vector generating module 2, the lower body vector generating module 3, and the sheet motion vector generating module 4, the sheet motion vector output by inputting the above learning data and the pre-labeled sheet motion vector value are and update parameters of the upper body vector generating module 2 , the lower body vector generating module 3 and the seat movement vector generating module 4 based on the difference.

With respect to the reasoning session of the upper body vector generating module 2, the lower body vector generating module 3 and the seat movement vector generating module 4, in the infotainment system of the vehicle, the user has both hands to hold both sides of the steering wheel and the right foot requests to maintain the reference state that is placed on the brake pedal, and transmits the user's upper body pressure distribution data 100 and lower body pressure distribution data 200 to the upper body vector generation module 2, the lower body vector generation module 3 and After input to the sheet movement vector generation module 4 to output the sheet movement vector, the sheet control determination module 5 determines the sheet movement vector to move the sheet by the sheet movement vector through the compiler 6 and the sheet control unit do.

Accordingly, there is an effect that the seat position is automatically set according to the user's body type, even if the user's body type is not input in advance or the seat position is not preset according to the user's body type.

At this time, the ConvNet included in the upper body vector generating module 2 and the lower body vector generating module 3 is composed of a weight and a bias that can be learned by each neuron like a general artificial neural network. Each neuron receives an input, performs a dot product, and then performs a non-linear operation according to the selection. The whole network has one score function that is differentiable like a general neural network. The raw image is read from the input layer and a score for each class is obtained from the output layer. In addition, ConvNet has the same loss function as SVM/Softmax in the last layer, and various techniques used when training a general neural network can be equally applied.

Unlike a general artificial neural network, the ConvNet architecture included in the upper body vector generating module 2 and the lower body vector generating module 3 can encode properties of matrix data thanks to the assumption that the input data is a matrix (or matrix). Such an architecture can implement a forward function more effectively and can greatly reduce the number of parameters required to train a network using matrix data.

The ConvNet included in the upper body vector generating module 2 and the lower body vector generating module 3 transforms the input vector through a series of hidden layers. Each hidden layer consists of neurons, and each neuron is connected to all neurons in the previous layer. This is called "fully connected". Neurons within the same layer do not have any connections and are independent of each other. The last fully-connected layer is called an output layer, and in the classification model, it represents a class score for which class it belongs to among a plurality of classes.

Unlike the ConvNet included in the upper body vector generation module (2) and the lower body vector generation module (3), a general artificial neural network is not suitable for handling matrices. For CIFAR-10 data, each matrix is 32x32x3 (32 horizontal and vertical, 3 color channels), so for one neuron in the first hidden layer, 32x32x3=3072 weights are needed, but using a larger matrix It is impossible to use the same structure for For example, a matrix of size 200x200x3 requires 200x200x3 = 120,000 weights for the same neuron. In addition, since there are a plurality of these neurons in the layer, the number of parameters increases significantly. As such, Fully-connectivity is a waste of money, and a large number of parameters leads to overfitting.

The ConvNet included in the upper body vector generating module 2 and the lower body vector generating module 3 can be structured in a more rational direction by taking advantage of the feature that the input is made up of a matrix. In particular, unlike general neural networks, ConvNet layers have three dimensions: horizontal, vertical, and depth. The depth referred to here is not the depth of the entire neural network, but the third dimension in the activation volume. For example, the CIFAR-10 matrix can be viewed as an input activation volume having a dimension of 32x32x3 (width, length, depth). In ConvNet architecture, neurons located in one layer are connected to only a part of the neurons in the previous layer, not all neurons in the previous layer, unlike the general neural network. Since the ConvNet architecture makes the entire matrix into a single vector of class scores, the final output layer has a dimension of 1x1x10 (10 is the number of classes in CIFAR-10 data).

Each layer of ConvNet included in the upper body vector generating module 2 and the lower body vector generating module 3 transforms one activation volume into another activation volume through a differentiable transform function. In the ConvNet architecture, three types of layers are mainly used: a convolutional layer, a pooling layer, and a fully-connected layer. The entire ConvNet architecture is built by stacking these three types of layers.

4 and 5 are exemplary views of ConvNet included in the upper body vector generating module 2 and the lower body vector generating module 3 according to an embodiment of the present invention. As shown in Figures 4 and 5, taking a simple ConvNet as an example, it can be constructed as [INPUT-CONV-RELU-POOL-FC]. When the upper body pressure distribution data 100 or the lower body pressure distribution data 200 is an input vector, when the INPUT input matrix has 32 horizontal, 32 vertical, and RGB channels, the size of the input may be [32x32x3]. The CONV layer (Conv. Filter) is connected to a partial area of the input matrix, and a dot product of the connected area and its own weight is calculated. The resulting volume will have a size equal to [32x32x12]. The RELU layer is an activation function applied to each element, such as max(0,x). The RELU layer does not change the size of the volume ([32x32x12]). As a result, Activation map 1 is created. The POOL layer (pooling) outputs a reduced volume (Activation map 2) as [16x16x12] by performing downsampling on the "horizontal, vertical" dimensions. The FC (fully-connected) layer calculates the class scores and outputs a pressure distribution vector volume (output layer) with a size of [1x1xn]. The FC layer is connected to all elements of the previous volume.

In this way, the ConvNet included in the upper body vector generating module 2 and the lower body vector generating module 3 transforms the original matrix composed of pixel values into class scores for distribution through each layer. Some layers have parameters, some layers have no parameters. In particular, CONV/FC layers are activation functions that include not only input volume but also weight and bias. On the other hand, RELU/POOL layers are fixed functions. The parameters of the CONV/FC layer are learned by gradient descent so that the class score for each matrix is equal to the label of the corresponding matrix.

The parameters of the CONV layer of ConvNet included in the upper body vector generating module 2 and the lower body vector generating module 3 consist of a series of learnable filters. Each filter is small in the horizontal/vertical dimension, but covers the entire depth in the depth dimension. In the forward pass, each filter is slid in the horizontal/vertical dimension of the input volume (convolve precisely), and a two-dimensional activation map is generated. When sliding the filter over the input, a dot product is made between the filter and the input volume. Through this process, ConvNet learns a filter that activates on a specific pattern at a specific location in the input data. Stacking these activation maps in the depth dimension becomes the output volume. Therefore, each element of the output volume treats only a small area of the input, and the neurons in the same activation map share the same parameters because they are the result of applying the same filter.

Examples of network structures that can be used in ConvNet included in the upper body vector generating module 2 and the lower body vector generating module 3 are as follows.

LeNet. The first successful ConvNet applications were created by Yann LeCun in the 1990s. Among them, the LeNet architecture that reads zip codes or numbers is the most famous.

AlexNet. What made ConvNet famous in the field of computer vision is AlexNet by Alex Krizhevsky, Ilya Sutskever, and Geoff Hinton. AlexNet participated in the ImageNet ILSVRC challenge 2012 and won 1st place by a big margin (top 5 error rate 16%, 2nd place 26%). The architecture is basically similar to LeNet, but deeper and larger. Also, unlike in the past, a POOL layer was stacked immediately after one CONV layer, it was configured in a way that several CONV layers were stacked.

ZF Net. The winners of the ILSVRC 2013 were created by Matthew Zeiler and Rob Fergus. It is called ZFNet after the authors. It was created by modifying hyperparameters such as adjusting the size of the intermediate CONV layer in AlexNet.

GoogLeNet. The winner of ILSVRC 2014 was Szegedy et al. This was made by Google. The biggest contribution of this model is to propose an Inception module that greatly reduces the number of parameters (4M, 60M for AlexNet). In addition, at the end of the ConvNet, average pooling is used instead of the FC layer to reduce a lot of seemingly insignificant parameters.

VGGNet. The second-placed network in ILSVRC 2014 is a model called VGGNet created by Karen Simonyan and Andrew Zisserman. The biggest contribution of this model is to show that the depth of the network is a very important factor for good performance. The best of the several models they propose consists of 16 CONV/FC layers, with all convolutions 3x3 and all pooling only 2x2. Although the matrix classification performance is slightly lower than that of GoogLeNet, it was later found that it performs better in several transfer learning tasks. Therefore, VGGNet has been used most recently for matrix feature extraction. The disadvantage of VGGNet is that it uses a lot of memory (140M) and requires a lot of computation.

ResNet. Residual Network created by Kaiming He et al. won the ILSVRC 2015. It uses a unique structure called skip connection and has a characteristic that batch normalizatoin is used a lot. This architecture does not use the FC layer in the last layer.

As described above, those skilled in the art to which the present invention pertains will be able to understand that the present invention may be embodied in other specific forms without changing the technical spirit or essential features thereof. Therefore, the above-described embodiments are to be understood in all respects as illustrative and not restrictive. The scope of the present invention is indicated by the following claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts should be construed as being included in the scope of the present invention.

The features and advantages described herein are not all inclusive, and many additional features and advantages will become apparent to those skilled in the art, particularly upon consideration of the drawings, the specification, and the claims. Moreover, it should be noted that the language used herein has been selected primarily for readability and teaching purposes, and may not be selected to delineate or limit the subject matter of the present invention.

The foregoing description of embodiments of the invention has been presented for purposes of illustration. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Those skilled in the art will appreciate that many modifications and variations are possible in light of the above disclosure.

Therefore, the scope of the present invention is not limited by the detailed description, but by any claims of the application based thereon. Accordingly, the disclosure of the embodiments of the present invention is illustrative and not intended to limit the scope of the present invention as set forth in the following claims.

1: Artificial neural network device for seat position automation
2: Upper body vector generation module
3: Lower body vector generation module
4: sheet movement vector generation module
5: Seat control judgment module
6: Compiler
10: upper pressure distribution sensor
20: lower pressure distribution sensor
100: upper body pressure distribution data
200: lower body pressure distribution data
300: sheet control data

Claims (5)

An artificial neural network module in which the first CNN module and the first LSTM module that use the time series upper body pressure distribution data output from the upper pressure distribution sensor mounted on the upper cushion of the car seat as input data and the upper body vector as output data are continuously combined an upper body vector generation module comprising;
A type of artificial neural network in which a second CNN module and a second LSTM module that use time series lower body pressure distribution data output from the lower pressure distribution sensor mounted on the lower cushion of the car seat as input data and lower body vectors as output data are continuously combined a lower body vector generation module comprising a module; and
A sheet motion vector generation module including an artificial neural network module in which the 3rd LSTM module, in which the upper body vector and lower body vector are concatenated, as input data, and the sheet motion vector as output data, and Softmax are continuously combined ;
including,
The seat movement vector is characterized in that it consists of a dimension corresponding to the number of movable directions of the car seat,
Artificial neural network device for seat position automation.
According to claim 1,
a sheet control determination module for determining whether to proceed with sheet control or to stop sheet control based on the magnitude of the sheet movement vector;
characterized in that it further comprises,
Artificial neural network device for seat position automation.
3. The method of claim 2,
a compiler that compiles the sheet movement vector into a language of the sheet control unit when the sheet control determination module determines to perform sheet control;
characterized in that it further comprises,
Artificial neural network device for seat position automation.
According to claim 1,
In the learning session of the upper body vector generating module, the lower body vector generating module, and the seat movement vector generating module, the user's upper body pressure distribution data with both hands gripping both sides of the steering wheel and the right foot resting on the brake pedal; Characterized by using the lower body pressure distribution data as a reference,
Artificial neural network device for seat position automation.
An artificial neural network module in which the first CNN module and the first LSTM module that use the time series upper body pressure distribution data output from the upper pressure distribution sensor mounted on the upper cushion of the car seat as input data and the upper body vector as output data are continuously combined an upper body vector generating step in which the upper body vector generating module including: receiving the upper body pressure distribution data as input data and outputting the upper body vector as output data;
An artificial neural network module in which a second CNN module and a second LSTM module that use time series lower body pressure distribution data output from the lower pressure distribution sensor mounted on the lower cushion of the car seat as input data and lower body vectors as output data are continuously combined a lower body vector generating step in which a lower body vector generating module comprising: receiving the lower body pressure distribution data as input data and outputting the lower body vector as output data; and
There is a sheet movement vector generation module including a 3rd LSTM module that uses a joint vector in which an upper body vector and a lower body vector are concatenated as input data and a sheet movement vector as output data, and an artificial neural network module in which Softmax is continuously combined. , a sheet movement vector generating step of receiving the combined vector as input data and outputting the sheet movement vector as output data;
including,
The seat movement vector is characterized in that it consists of a dimension corresponding to the number of movable directions of the car seat,
A method of automating seat position using an artificial neural network.

Images