WO2021115268A1 - Method and apparatus for determining running direction of metro train, and terminal and storage medium - Google Patents

Abstract

Disclosed are a method and apparatus for determining a running direction of a metro train, and a terminal and a storage medium, wherein same belong to the field of artificial intelligence. The method comprises: at a metro train acceleration stage, acquiring first acceleration data collected by a linear acceleration sensor, wherein the first acceleration data comprises acceleration data of all directions under a terminal coordinate system; converting the first acceleration data into second acceleration data, wherein the second acceleration data comprises acceleration data of all directions under a world coordinate system; and determining the running direction of a metro train according to the second acceleration data within a target time range. The first acceleration data collected by the terminal is converted into second acceleration data, such that the attitude of the terminal is prevented from affecting the acquisition of an acceleration direction; and the second acceleration data within the target time range is comprehensively analyzed, such that the problem of the determined running direction of a metro train being inaccurate caused due to the instability of a single acceleration direction can be avoided.

Description

Method, device, terminal and storage medium for determining subway running direction

This application claims the priority of the Chinese patent application filed on December 10, 2019 with the application number 201911259924.2 and the invention title "Method, device, terminal and storage medium for determining the direction of subway operation", the entire content of which is incorporated by reference In this application.

Technical field

The embodiments of the present application relate to the field of artificial intelligence, and in particular to a method, device, terminal, and storage medium for determining the direction of subway operation.

Background technique

As a daily means of transportation, the subway can provide people with convenient and fast rail transit services.

Due to the directional nature of the subway operation, when taking the subway, people need to choose the subway in the corresponding direction according to the destination station and the current station. However, the current running direction of the subway requires human judgment. Correspondingly, when the running direction of the subway is wrong, the passengers are required to make a transfer after self-perception.

Summary of the invention

The embodiments of the present application provide a method, device, terminal, and storage medium for determining the direction of subway operation. The technical solution is as follows:

On the one hand, an embodiment of the present application provides a method for determining a subway running direction, the method is used in a terminal, and the method includes:

During the subway acceleration phase, acquiring first acceleration data collected by the linear acceleration sensor, where the first acceleration data includes acceleration data in various directions in the terminal coordinate system;

Converting the first acceleration data into second acceleration data, where the second acceleration data includes acceleration data in various directions in the world coordinate system;

Determine the running direction of the subway according to the second acceleration data within the target duration.

On the other hand, an embodiment of the present application provides a device for determining the direction of subway operation. The device is used in a terminal, and the device includes:

The acquiring module is configured to acquire the first acceleration data collected by the linear acceleration sensor during the subway acceleration phase, where the first acceleration data includes acceleration data in various directions in the terminal coordinate system;

A data conversion module, configured to convert the first acceleration data into second acceleration data, the second acceleration data including acceleration data in various directions in the world coordinate system;

The first determining module is configured to determine the running direction of the subway according to the second acceleration data within the target duration.

On the other hand, an embodiment of the present application provides a terminal, the terminal includes a processor and a memory; the memory stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the above-mentioned aspects. The method for determining the running direction of the subway.

In another aspect, a computer-readable storage medium is provided, the storage medium stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the method for determining a subway running direction as described in the above aspect.

On the other hand, the embodiments of the present application provide a computer program product or computer program. The computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the terminal reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the terminal executes the method for determining the subway running direction provided in the various optional implementations of the foregoing aspects.

Description of the drawings

Fig. 1 shows a flowchart of a method for determining a subway running direction provided by an exemplary embodiment of the present application;

Fig. 2 shows a flowchart of a method for determining a subway running direction shown in an exemplary embodiment of the present application;

Fig. 3 is a schematic diagram of a terminal coordinate system and a world coordinate system shown in an exemplary embodiment of the present application;

Fig. 4 shows a flowchart of a method for determining a subway running direction shown in another exemplary embodiment of the present application;

Fig. 5 is a schematic diagram showing the direction of gravity and the direction of magnetic force on the earth according to an exemplary embodiment of the present application;

Fig. 6 is a framework diagram of a running direction prediction model shown in an exemplary embodiment of the present application;

Fig. 7 shows a flowchart of a method for determining a subway running direction shown in another exemplary embodiment of the present application;

FIG. 8 shows a flowchart of a process of determining that a subway is in an acceleration phase shown in an exemplary embodiment of the present application;

FIG. 9 is a schematic diagram of a process of audio data preprocessing according to an exemplary embodiment of the present application;

FIG. 10 shows a flow chart of a method for determining whether a user takes the subway correctly according to an exemplary embodiment of the present application;

Fig. 11 shows a structural block diagram of a device for determining a subway running direction provided by an exemplary embodiment of the present application;

FIG. 12 shows a structural block diagram of a terminal provided by an exemplary embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions, and advantages of the present application clearer, the implementation manners of the present application will be described in further detail below in conjunction with the accompanying drawings.

The "plurality" mentioned herein means two or more. "And/or" describes the association relationship of the associated objects, indicating that there can be three types of relationships, for example, A and/or B, which can mean: A alone exists, A and B exist at the same time, and B exists alone. The character "/" generally indicates that the associated objects before and after are in an "or" relationship.

In related technologies, two methods are usually used to determine the direction of the subway. One is to determine the direction of the subway by the direction of the instantaneous acceleration: when the user rides the subway, the acceleration sensor in the terminal can record the instantaneous acceleration and pass the instantaneous acceleration. The direction of acceleration determines the running direction of the subway at this time; the other is to determine the running direction of the subway through the compass (electronic compass) direction.

Obviously, using the method of determining the direction of subway operation in related technologies, on the one hand, the acceleration direction recorded by the acceleration sensor in the terminal will be affected by the terminal posture, that is, the user holding the terminal in different postures will cause the instantaneous acceleration detected by the terminal There are differences in directions; and if the user is moving in the subway, the instantaneous acceleration collected by the acceleration sensor in the terminal includes not only the acceleration of the subway, but also the acceleration generated by the user’s movement on the terminal; in addition, the subway will produce a certain amount of acceleration during the acceleration process. The oscillating phenomenon makes the accuracy of determining the direction of subway operation only by using a certain instantaneous acceleration direction, and the accuracy is low. On the other hand, using the electronic compass to determine the direction of the subway, because the subway operation will affect the accuracy of the electronic compass, therefore, directly using the electronic compass to determine the direction of the subway, the accuracy is low.

In order to solve the above-mentioned problem, an embodiment of the present application provides a method for determining the running direction of a subway, and the method includes a process as shown in FIG.

Step 101: Obtain linear acceleration in the terminal coordinate system.

Step 102: Convert the linear acceleration in the terminal coordinate system to the acceleration in the world coordinate system through the coordinate system conversion module.

Step 103: Acquire acceleration data in the world coordinate system for a period of time.

A period of time can be the acceleration phase of the subway, and the acceleration data converted into the world coordinate system corresponding to the acceleration phase of the subway is cached.

Step 104: Input the acquired acceleration data in the world coordinate system for a period of time into the running direction prediction model.

The acceleration data buffered for a period of time is input into the running direction prediction model, and the prediction probability of each candidate direction can be output.

Step 105: Determine the subway running direction according to the predicted probability of each candidate running direction.

The candidate direction with the highest predicted probability can be determined as the subway running direction.

The method for determining the running direction of the subway adopted in the embodiment of this application converts the linear acceleration in the terminal coordinate system acquired by the terminal into the acceleration in the world coordinate system. Due to the uniqueness of the world coordinate system, the terminal attitude pair acquisition is avoided. The impact of the acceleration direction obtained, thereby improving the accuracy of determining the direction of the subway; in addition, by obtaining the acceleration data in the world coordinate system for a period of time, and inputting it into the prediction model of the direction of travel, in order to determine the direction of the subway for a period of time. Comprehensive analysis of the acceleration data to determine the direction of the subway can avoid the problem of inaccurate determination of the direction of the subway due to the instability of a single acceleration direction.

Please refer to FIG. 2, which shows a flowchart of a method for determining a subway running direction shown in an exemplary embodiment of the present application. In this embodiment, the method for determining the running direction of the subway is used in a terminal as an example for description, and the method includes:

Step 201: In the subway acceleration phase, first acceleration data collected by the linear acceleration sensor is acquired, and the first acceleration data includes acceleration data in various directions in the terminal coordinate system.

The terminal is provided with a linear acceleration sensor, and the data acquired by the linear acceleration sensor is acceleration data after excluding the influence of gravity, and is acceleration data in the terminal coordinate system.

In a possible implementation, when the terminal recognizes that the subway is in an acceleration phase, the linear acceleration sensor collects acceleration data in various directions in the terminal coordinate system to obtain the first acceleration data in the terminal coordinate system. For example, the first acceleration data can be expressed as a vector as

Where x _α is the acceleration component in the X-axis direction in the _{terminal coordinate system, y α} is the acceleration component in the Y-axis direction in the _{terminal coordinate system, and z α} is the acceleration component in the Z-axis direction in the terminal coordinate system.

Illustratively, the terminal coordinate system may be as shown in (A) in FIG. 3, and the terminal coordinate system changes with the posture of the terminal.

Step 202: Convert the first acceleration data into second acceleration data, where the second acceleration data includes acceleration data in various directions in the world coordinate system.

Since the terminal posture will change with the user’s holding posture, the terminal coordinate system will also change with the user’s holding posture, which causes the acceleration direction corresponding to the first acceleration data to change at any time. The acceleration direction determines the reference value of the subway running direction. Therefore, the first acceleration data needs to be converted into the corresponding second acceleration data in the world coordinate system, and the world coordinate system will not be affected by the terminal posture.

Illustratively, the world coordinate system may be as shown in (B) of FIG. 3, where the Z axis in the world coordinate system is perpendicular to the ground upward, the Y axis points to the north, and the X axis points to the east.

In a possible implementation manner, the first acceleration data is converted into the second acceleration data in the world coordinate system according to the data conversion relationship between the world coordinate system and the terminal coordinate system. For example, if the relationship between the second acceleration data and the first acceleration data is

among them,

Represents the second acceleration data, R represents the data conversion relationship between the first acceleration data and the second acceleration data,

Indicates the first acceleration data; according to the first acceleration data

And data conversion relationship R, where x _α is the acceleration component of the first acceleration data in the X-axis direction in the _{terminal coordinate system, and y α} is the acceleration component of the first acceleration data in the Y-axis direction in the terminal coordinate system, z _α is the acceleration component of the first acceleration data in the Z-axis direction in the terminal coordinate system; the second acceleration data can be obtained

Among them, x _β is the acceleration component of the second acceleration data in the X-axis direction in the _{world coordinate system, y β} is the acceleration component of the second acceleration data in the Y-axis direction in the _{world coordinate system, and z β} is the world coordinate system Next, the acceleration component of the second acceleration data in the Z-axis direction.

Step 203: Determine the running direction of the subway according to the second acceleration data within the target duration.

Since the acceleration direction of a single point of the subway is very unstable during the acceleration phase, the overall direction of the acceleration is stable during the entire acceleration time period. Therefore, the embodiment of the present application uses all the acceleration directions during the subway acceleration time period to synthesize Determine the direction of the subway.

Among them, the target duration is preset by the developer, and the target duration may be determined by the developer by comprehensively analyzing the acceleration time in actual subway operation. Optionally, the target duration may be the acceleration duration when the subway starts, or it may be a period of time in the acceleration duration.

In a possible implementation manner, the terminal continuously collects the first acceleration data within the target duration, converts it into the second acceleration data, and stores it in the terminal, so that the acceleration data in the target duration can be subsequently comprehensively analyzed. For example, the target duration may be 10s, that is, the terminal continuously collects the first acceleration data within 10s, and converts it into second acceleration data in real time, so as to comprehensively analyze the subway running direction based on the second acceleration data within 10s.

In a possible implementation manner, the terminal performs a comprehensive analysis according to the second acceleration data buffered to obtain the overall acceleration direction corresponding to the second acceleration data within the target duration, and determines the acceleration direction as the subway running direction.

To sum up, in the embodiment of the present application, when the subway is in the acceleration phase, the terminal collects acceleration data (first acceleration data) in various directions in the terminal coordinate system through the linear acceleration sensor, and converts the first acceleration data. Obtain acceleration data (second acceleration data) in each direction in the world coordinate system, so as to determine the subway running direction according to the second acceleration data within the target duration. By converting the first acceleration data collected by the terminal into the second acceleration data, due to the uniqueness of the world coordinate system, the influence of the terminal attitude on the acquired acceleration direction is avoided, thereby improving the accuracy of determining the direction of subway operation. In addition, , Through the comprehensive analysis of the second acceleration data within the target duration to determine the direction of the subway, the problem of inaccurate determination of the direction of the subway due to the instability of a single acceleration direction can be avoided.

Optionally, a gravity acceleration sensor and a magnetometer are provided in the terminal;

Converting the first acceleration data to the second acceleration data includes:

Obtain the gravity acceleration data collected by the gravity acceleration sensor and the magnetic data collected by the magnetometer. The gravity acceleration data and the magnetic data are based on the terminal coordinate system;

Determine the conversion matrix according to the gravity acceleration data and the magnetic data, and the conversion matrix is used to convert the acceleration data in the terminal coordinate system into the acceleration data in the world coordinate system;

Through the conversion matrix, the first acceleration data is converted into the second acceleration data.

Optionally, linear acceleration sensors, gravity acceleration sensors and magnetometers collect data according to the target sampling frequency;

Obtain the gravity acceleration data collected by the gravity acceleration sensor and the magnetic force data collected by the magnetometer, including:

Obtain the nth gravitational acceleration data collected by the gravitational acceleration sensor and the nth magnetic force data collected by the magnetometer, where n is an integer greater than or equal to 2;

Determine the conversion matrix based on the gravitational acceleration data and the magnetic force data, including:

If the similarity between the nth gravitational acceleration data and the n-1th gravitational acceleration data is greater than the similarity threshold, and the similarity between the nth magnetic data and the n-1th magnetic data is greater than the similarity threshold, then the n-1th conversion matrix Determined as the nth conversion matrix, and the n-1th conversion matrix is determined according to the n-1th gravitational acceleration data and the n-1th magnetic force data;

If the similarity between the nth gravitational acceleration data and the n-1th gravitational acceleration data is less than the similarity threshold, and/or the similarity between the nth magnetic data and the n-1th magnetic data is less than the similarity threshold, then according to the nth gravity The acceleration data and the n-th magnetic force data determine the n-th conversion matrix;

Converting the first acceleration data to the second acceleration data through the conversion matrix includes:

Through the nth conversion matrix, the first acceleration data is converted into the second acceleration data.

Optionally, determining the direction of subway operation according to the second acceleration data within the target duration includes:

Input the second acceleration data within the target duration into the running direction prediction model to obtain the running direction prediction result output by the running direction prediction model. The running direction prediction result includes the predicted probability corresponding to each candidate running direction;

According to the predicted probability of each candidate running direction, the subway running direction is determined.

Optionally, the running direction prediction model adopts the convolutional neural network CNN structure, and the running direction prediction model is trained based on the sample acceleration data and the sample running direction;

The sample acceleration data is the acceleration data in all directions in the world coordinate system during the acceleration phase of the subway, and the sample running direction is the actual running direction of the subway.

Optionally, after determining the subway running direction according to the second acceleration data within the target duration, the method further includes:

Determine the end station based on the current station and the target station, and the target station is located between the current station and the end station;

According to the current station and the terminal station, find the target subway running direction from the database. The target subway running direction is the running direction when the subway runs from the current station to the terminal station;

If the determined subway running direction does not match the target subway running direction, a transfer prompt will be given.

Optionally, the subway acceleration phase includes the subway startup phase;

In the subway acceleration stage, before acquiring the first acceleration data collected by the linear acceleration sensor, the method further includes:

Collect ambient sound through a microphone;

When it is recognized that the ambient sound includes the subway closing alarm bell, it is determined that the subway is in the start-up phase.

Optionally, when it is recognized that the ambient sound includes the subway closing alarm bell, it is determined that it is in the subway startup stage, including:

Perform audio feature extraction on environmental sounds to obtain an audio feature matrix;

Input the audio feature matrix into the voice recognition model to obtain the alarm ringtone recognition result output by the voice recognition model. The alarm ringtone recognition result is used to indicate whether the ambient sound contains the subway closing alarm ringtone;

If the alarm bell recognition result indicates that the ambient sound includes a subway closing alarm bell, it is determined that it is in the subway startup stage.

Optionally, determining the conversion matrix according to the gravity acceleration data and the magnetic force data includes:

According to the gravitational acceleration data and the magnetic force data, determine the target vector corresponding to the unit vector in each direction in the world coordinate system in the terminal coordinate system;

According to the unit vector and the target vector, the conversion matrix between the world coordinate system and the terminal coordinate system is determined.

In a possible implementation, in order to improve the accuracy of determining the running direction of the subway based on the second acceleration data, a prediction model of the running direction can be preset in the terminal, and the second acceleration data can be used as the model input, so as to output each candidate The predicted probability of the running direction is used to judge the current running direction of the subway.

Please refer to FIG. 4, which shows a flowchart of a method for determining a subway running direction shown in another exemplary embodiment of the present application. In this embodiment, the method for determining the running direction of the subway is used in a terminal as an example for description, and the method includes:

Step 401: In the subway acceleration phase, first acceleration data collected by the linear acceleration sensor is acquired.

For the implementation of this step, reference may be made to step 201, which is not described in detail in this embodiment.

Step 402: Obtain the gravity acceleration data collected by the gravity acceleration sensor and the magnetic force data collected by the magnetometer. The gravity acceleration data and the magnetic force data are based on the terminal coordinate system.

Among them, the terminal is provided with a gravity acceleration sensor and a magnetometer. The gravity acceleration sensor collects gravity acceleration data in the terminal coordinate system, and the magnetometer collects magnetic force data in the terminal coordinate system.

In a possible implementation, the gravity acceleration data in the terminal coordinate system acquired by the gravity sensor can be expressed as

Among them, G _x is the acceleration component of the gravitational acceleration data in the X-axis direction, G _y is the acceleration component of the gravitational acceleration data in the Y-axis direction, and G _z is the acceleration component of the gravitational acceleration data in the Z-axis direction; The magnetic data in the terminal coordinate system can be expressed as

Wherein, M _x is a magnetic data magnetic field component data in the Y-axis direction, M _z for the component of the magnetic field data in the Z-axis direction magnetic field component in the X-axis direction, M _y.

Schematically, a schematic diagram of the gravity and magnetic force of the earth is shown in Figure 5, where G is vertically downward, that is, the direction of the Z axis in the world coordinate system shown in (B) in Figure 3 is opposite; due to the magnetic induction The direction of the line is from the south pole of the earth to the north pole of the earth. Therefore, the direction of the magnetic force is shown as M in Figure 5.

It should be noted that step 401 and step 402 can be performed at the same time, and there is no sequence, that is, when it is determined that the subway is in the acceleration phase, the linear acceleration sensor, the gravity acceleration sensor, and the magnetometer in the terminal start to collect data at the same time.

Step 403: Determine a conversion matrix according to the gravity acceleration data and the magnetic force data, and the conversion matrix is used to convert acceleration data in the terminal coordinate system into acceleration data in the world coordinate system.

In a possible implementation manner, the process of determining the conversion matrix according to the gravitational acceleration data and the magnetic force data may include the following steps:

1. Determine the target vector corresponding to the unit vector in each direction in the world coordinate system in the terminal coordinate system according to the gravitational acceleration data and the magnetic force data.

In the embodiment of the present application, the conversion relationship between the first acceleration data and the second acceleration data collected by the terminal is determined based on the representation relationship of the unit vector in the world coordinate system and the terminal coordinate system. Since the unit vector in the world coordinate system is related to the gravitational acceleration and magnetic force data, for example, the direction of the Z-axis unit vector in the world coordinate system is opposite to the gravitational acceleration, and the corresponding gravity acceleration data can be used to represent the Z-axis unit vector in the world coordinate system. Therefore, in a possible implementation manner, the target vector corresponding to the unit vector in each direction in the world coordinate system in the terminal coordinate system can be determined according to the gravitational acceleration data and the magnetic force data collected by the terminal.

Schematically, since the gravitational acceleration data obtained by the gravity sensor is opposite to the unit vector of the Z axis in the world coordinate system, according to

It can be obtained that the Z-axis unit vector of the world coordinate system is expressed in the terminal coordinate system as

Optionally, as can be seen from (B) in Figure 5 and Figure 3,

with

The cross product of can be obtained perpendicular to

with

Vector of the plane composed

And the vector

The direction of is pointing to the west in the world coordinate system, which is opposite to the unit vector of the X axis shown in (B) in Figure 3. Therefore, the unit vector of the X axis of the world coordinate system can be expressed in the terminal coordinate system as:

Among them, H _x , H _y , and H _z are the unit vectors of the X axis of the world coordinate system, respectively, corresponding to the components in the X, Y, and Z directions of the terminal coordinate system.

Optionally, the X-axis unit vector in the known world coordinate system

And the Z-axis unit vector

Since the Y axis is perpendicular to the plane formed by the X axis and the Z axis, according to the vector

And vector

You can get the unit vector of the Y axis

In a possible implementation, the obtained three-axis unit vectors (unit vectors) of the world coordinate system are expressed in the terminal coordinate system (target vectors) as:

2. Determine the conversion matrix between the world coordinate system and the terminal coordinate system according to the unit vector and the target vector.

Since the unit vector is a vector in the world coordinate system, and the target vector is the representation of the same unit vector in the terminal coordinate system, that is, the unit vector and the target vector are the representation of the same vector in different coordinate systems. In a possible implementation manner, the conversion relationship between the world coordinate system and the terminal coordinate system can be calculated according to the relationship between the unit vector and the target vector, and the same acceleration data in the terminal coordinate system and the world coordinate system can be correspondingly obtained. The conversion relationship.

Schematically, suppose there is a transformation matrix R and a point in the terminal coordinate system

(I.e. the representation of the unit vector in the terminal coordinate system) and a point in the world coordinate system

(That is, the representation of the unit vector in the world coordinate system), then:

due to

They are the unit vectors of the X-axis, Y-axis and Z-axis in the world coordinate system, so there are:

Put the above vectors into formula (1) respectively to get:

Converted to matrix form:

due to

Is the identity matrix E, the formula (2) can be expressed as:

R·D=E (3)

among them,

Since D is an orthogonal matrix, that is, D ^-1 =D ^T , therefore:

Among them, R is the conversion matrix. From the formula (4), it can be known that the conversion matrix R is related to gravitational acceleration data and magnetic force data. Therefore, in a possible implementation manner, when the terminal collects the first acceleration data through the linear acceleration sensor, the corresponding gravitational acceleration and magnetic force data are respectively acquired through the gravity acceleration sensor and the magnetometer, and the corresponding gravitational acceleration and magnetic force data are obtained according to the above conversion matrix R and gravity. According to the relationship between the acceleration value and the magnetic data, the conversion matrix R corresponding to the first acceleration data at this time is obtained.

Step 404: Convert the first acceleration data into second acceleration data through the conversion matrix.

In a possible implementation manner, after the terminal obtains the conversion matrix corresponding to the first acceleration data, the first acceleration data and the conversion matrix are brought into formula (1), and the world coordinate system corresponding to the first acceleration data can be obtained. Under the second acceleration data.

Illustratively, if the first acceleration data is denoted as L_ACC _α , and the conversion matrix corresponding to the first acceleration data is R, then according to formula (1), it can be obtained that the second acceleration data is L_ACC _β =R·L_ACC _α .

Step 405: Input the second acceleration data within the target duration into the running direction prediction model to obtain the running direction prediction result output by the running direction prediction model. The running direction prediction result includes the predicted probability corresponding to each candidate running direction.

In a possible implementation manner, the running direction prediction model adopts a convolutional neural network (Convolutional Neural Network, CNN) structure, and the training process of the model is as follows:

1. Collect training sample data.

Among them, the training sample data includes the sample acceleration data and the sample running direction. The sample acceleration data is the acceleration data of the subway in various directions under the world coordinate system, and the sample running direction is the actual running direction of the subway.

In a possible implementation manner, the developer collects the acceleration data of the subway acceleration phase of each subway station in advance. Since the acceleration data is the acceleration data in the terminal coordinate system, in order to avoid the influence of the terminal attitude on the acceleration direction, it is The steps described above are converted into acceleration data in the world coordinate system, and then the acceleration data is marked according to the actual running direction of the subway at the station. For example, the actual running direction of the subway is divided into southeast, southwest, northeast, Northwest, etc., and each actual operating direction is marked as 0, 1, 2, 3, etc., that is, 0 represents the southeast direction, 1 represents the southwest direction, 2 represents the northeast direction, and 3 represents the northwest direction.

Optionally, the actual running direction of the subway at the station can be obtained according to map applications.

Optionally, the actual running direction of the subway may also include east, south, west, north, etc., or a more detailed division, for example, 30° from north to east. The embodiment of the present application does not limit the representation of the running direction of the sample.

2. Construct a CNN model.

In a possible implementation manner, the CNN model structure is shown in FIG. 6, the first convolutional layer 601 and the second convolutional layer 602 are used to extract the characteristics of the input sample acceleration data, and the first fully connected layer 603 and the second convolutional layer The second fully connected layer 604 integrates the category-discriminatory information in the convolutional layers 601 and 602, and finally a normalized exponential function (Softmax) 605 is used to classify the integrated information of the fully connected layer to obtain the running direction prediction result.

Optionally, the loss function of the CNN model in this embodiment of the application may adopt a cross-entropy loss function, or other loss functions, such as focal loss function (Focal loss), the loss used for the prediction direction model in this embodiment of the application Functions do not constitute limitations.

3. Import the training samples into the CNN model for training.

In a possible implementation manner, a Tensorflow system can be used to train a CNN classification model, and a cross-entropy loss function and a gradient descent algorithm can be used until the model converges.

Optionally, the running direction prediction model can also use other traditional machine learning classifiers or deep learning classification models, which are not limited in this embodiment.

In a possible implementation, the buffered second acceleration data is input into the trained direction prediction model to obtain the prediction results of each running direction, that is, the prediction probability corresponding to each candidate running direction, where each candidate running direction is The actual running direction of the subway pre-calibrated during the model training process.

Illustratively, the prediction results of each running direction can be: P ₀ =0.01, P ₁ =0.005, P ₂ =0.98, P ₃ =0.005, etc., where 0 represents the southeast direction, 1 represents the southwest direction, and 2 represents the northeast direction , 3 represents the northwest direction, etc. Correspondingly, the prediction results include: the probability that the second acceleration data indicates that the subway direction is southeast is 0.01, the probability that the subway direction is southwest is 0.005, and the probability that the subway direction is northeast is 0.98, the probability of the subway running direction northwest is 0.005.

Step 406: Determine the subway running direction according to the predicted probability of each candidate running direction.

In a possible implementation, according to the predicted probability of each candidate running direction, the running direction with the highest predicted probability is determined as the subway running direction.

In an illustrative example, if the prediction result of the subway running direction is: P ₀ =0.01, P ₁ =0.005, P ₂ =0.98, P ₃ =0.005, where 0 represents the southeast direction, 1 represents the southwest direction, and 2 Represents the northeast direction and 3 represents the northwest direction. Since P ₂ >P ₀ >P ₁ =P ₃ , the subway running direction is determined to be the northeast direction.

In this embodiment, when the terminal collects the first acceleration data, the gravity sensor and the magnetometer are used to collect the gravity acceleration data and the magnetic force data respectively, so as to obtain the conversion matrix corresponding to the first acceleration data at this time; in addition, by pre-setting in the terminal Set the running direction prediction model, and input the cached second acceleration data into the running direction prediction model, and determine the subway running direction according to the output result of the running direction prediction model, which can improve the accuracy of determining the subway running direction based on the second acceleration data ; In addition, because the second acceleration data in the input running direction prediction model is acceleration data within a period of time, it can also avoid the problem of inaccurate determination of the subway running direction caused by the instability of a single acceleration direction, and further improve the determination of the subway running direction Accuracy.

Since the first acceleration data needs to be collected continuously, the attitude of the terminal may change during the collection process, causing the gravity acceleration data and magnetic force data to be changed, and due to the conversion matrix and the gravity acceleration data and magnetic force collected by the terminal in real time If the data is related, the corresponding gravitational acceleration and magnetic data changes will also cause the transformation matrix to change. If the same transformation matrix is used for different collection times, it is obvious that the change of the terminal attitude will cause the determined subway running direction to appear Therefore, in order to further improve the accuracy of the acquired second acceleration data, it is necessary to determine the conversion matrix in real time according to the gravity acceleration data and the magnetic force data, so that the determined conversion matrix conforms to the current holding posture of the terminal.

Illustratively, on the basis of FIG. 4, as shown in FIG. 7, steps 402 to 404 may be replaced with steps 407 to 410.

Step 407: Obtain the nth gravitational acceleration data collected by the gravitational acceleration sensor and the nth magnetic force data collected by the magnetometer, where n is an integer greater than or equal to 2.

In a possible implementation manner, when the terminal acquires the first acceleration data within the target duration, the linear acceleration sensor, the gravitational acceleration sensor, and the magnetometer in the terminal all perform data collection according to the target sampling frequency, so as to obtain the first acceleration data. The acceleration data, the gravitational acceleration data, and the magnetic force data are all data at the same time, so that the acquired second acceleration data has more reference value.

Illustratively, if the target duration is 10s and the sampling frequency is 20HZ, then the first acceleration data, gravitational acceleration data, and magnetic data acquired within the target duration are all 200, and the nth gravitational acceleration data is the nth acquisition The obtained gravitational acceleration data, the nth magnetic force data is the magnetic force data collected for the nth time.

Step 408: If the similarity between the nth gravitational acceleration data and the n-1th gravitational acceleration data is greater than the similarity threshold, and the similarity between the nth magnetic data and the n-1th magnetic data is greater than the similarity threshold, then the n-th 1 The conversion matrix is determined as the nth conversion matrix, and the n-1th conversion matrix is determined according to the n-1th gravitational acceleration data and the n-1th magnetic force data.

Since the conversion matrix corresponding to each collection time is related to the gravitational acceleration data and magnetic data collected at the current collection time, in a possible implementation manner, the gravitational acceleration data or magnetic data at adjacent collection time can be compared. , To determine whether the terminal posture has changed, and thus whether the conversion matrix has changed.

In a possible implementation manner, a similarity threshold is preset in the terminal, and if the terminal determines that the similarity between the nth reacceleration data and the n-1th gravitational acceleration data is greater than the similarity threshold, and the nth magnetic force data and the nth -1 The similarity of the magnetic data is greater than the similarity threshold, it is determined that the posture of the terminal has not changed significantly at the time of the two acquisitions. Therefore, there is no need to recalculate the conversion matrix, and the n-1th conversion matrix can be directly determined as the nth conversion matrix. In order to reduce the amount of calculation of the terminal and save power consumption.

Among them, the similarity calculation formula can be:

Among them, d is the similarity,

Is the nth gravitational acceleration data,

Is the n-1th gravitational acceleration data, convert d to a percentage, that is, obtain the similarity between the nth re-acceleration data and the n-1th gravitational acceleration data. The similarity between the nth magnetic force data and the n-1th magnetic force data can be obtained by referring to the above formula.

Illustratively, the similarity threshold may be 98%, that is, when the terminal determines that the similarity between the nth gravitational acceleration data and the n-1th gravitational acceleration data is 98.5%, the similarity between the nth magnetic force data and the n-1th magnetic force data If it is 99% and the similarity is greater than 98%, it is determined that the posture of the terminal has not changed significantly at the time of the two acquisitions, and the n-1th conversion matrix can be used continuously.

Step 409: If the similarity between the nth gravitational acceleration data and the n-1th gravitational acceleration data is less than the similarity threshold, and/or the similarity between the nth magnetic data and the n-1th magnetic data is less than the similarity threshold, then The nth gravitational acceleration data and the nth magnetic force data determine the nth conversion matrix.

In a possible implementation, if the similarity between the nth gravitational acceleration data and the n-1th gravitational acceleration data is less than the similarity threshold, or the similarity between the nth magnetic data and the n-1th magnetic data is less than the similarity Threshold, it is determined that the attitude of the terminal may change significantly at the time of the two collections, that is, the n-1th conversion matrix has no reference value during the nth collection of data. Therefore, it is necessary to re-based the nth gravitational acceleration data and the nth magnetic force. The data calculates the nth transformation matrix.

Illustratively, the similarity threshold may be 98%. If the terminal calculates that the similarity between the nth gravitational acceleration data and the n-1th gravitational acceleration data is 60%, which is less than the similarity threshold, the nth magnetic data and the nth -1 The similarity between the magnetic data is 99%, which is greater than the similarity threshold. It can be seen that the gravitational acceleration has changed significantly. Correspondingly, it is determined that the terminal attitude may change significantly at the time of the two acquisitions. The nth magnetic force data calculates the nth conversion matrix.

Step 410: Convert the first acceleration data into second acceleration data through the nth conversion matrix.

In a possible implementation manner, the first acceleration data collected for the nth time is converted into the nth second acceleration data through the determined nth conversion matrix.

In the embodiment of the present application, by comparing the similarity between the nth gravitational acceleration data and the n-1th gravitational acceleration data, and the similarity between the nth magnetic force data and the n-1th magnetic force data, the above similarity is greater than the similarity threshold. , You can directly convert the n-th first acceleration data to the n-th second acceleration data according to the n-1th conversion matrix, otherwise, you need to recalculate the n-th conversion matrix based on the n-th gravitational acceleration data and the n-th magnetic data. Performing data conversion can reduce the calculation amount of the terminal while ensuring the accuracy of the acquired second acceleration data.

Since the terminal needs to collect the first acceleration data when it is determined that the subway is in the acceleration phase, and usually the subway accelerates during the start-up phase, in order to improve the accuracy of the timing of collecting the first acceleration data, it is determined to determine the direction of the subway. In a possible implementation manner, the terminal can determine whether the subway is activated by identifying whether the ambient sound includes the subway closing alarm bell, so as to achieve accurate control of the timing of the first acceleration data collection.

Schematically, please refer to FIG. 8, which shows a flowchart of the process of determining that the subway is in the acceleration phase shown in an exemplary embodiment of the present application. This embodiment uses the method for determining the direction of the subway to be used in the terminal as an example for description. The method includes:

Step 801: Collect environmental sounds through a microphone.

In a possible implementation manner, the terminal can acquire user location information in real time, and when it is determined that the user enters the subway according to the user location information, the terminal can turn on the microphone to collect ambient sound in real time.

Optionally, when the user uses a payment application to swipe his card to take the subway, the terminal confirms that the user enters the subway, and can turn on the microphone to collect ambient sound in real time.

Optionally, in order to reduce the power consumption of the terminal, the terminal may use a low-power microphone to collect ambient sound in real time.

In step 802, when it is recognized that the ambient sound includes the subway door-closing alarm bell, it is determined that it is in the subway startup stage.

In a possible implementation, the terminal pre-stores the audio data of the subway closing alarm bell. After the terminal collects the ambient sound through the microphone in real time, it can convert the ambient sound into audio data and perform data processing on the audio data. , So as to identify whether the processed audio data contains the audio data corresponding to the subway closing alarm bell. If it is recognized that the ambient sound contains audio data corresponding to the subway door-closing alarm bell, it is determined that the subway is activated.

Optionally, the pre-stored subway closing alarm ringtone in the terminal can be the user turning on the microphone to collect the subway closing alarm ringtone and save it when taking the subway; or the terminal can obtain the subway corresponding to the subway when the terminal obtains the city's transportation route map. The door-closing alarm ringtone is converted into audio data and saved to the local.

Since the environmental sound may contain a variety of noises, in order to improve the recognition accuracy, it is necessary to preprocess the audio data corresponding to the environmental sound, and then input the processed audio data into the voice recognition model, so as to output the alarm bell according to the voice recognition model According to the recognition result, it is judged whether the current environmental sound includes the subway closing alarm bell.

In a possible implementation manner, the method for determining whether the subway closing alarm ring is included in the current ambient sound to determine whether the subway is activated may include the following steps:

1. Perform audio feature extraction on ambient sounds to obtain an audio feature matrix.

Since the voice recognition model cannot directly recognize audio data, it is necessary to pre-process the audio data to obtain digital features that can be recognized by the voice recognition model. Since the terminal microphone collects ambient sound in real time, its audio data is not stable as a whole, but its parts can be regarded as stable data, and the sound recognition model can only recognize the stable data, so the terminal first divides the corresponding audio data into frames Processing to obtain audio data corresponding to different audio frames.

In a possible implementation manner, the audio data pre-processing process may be as shown in Figure 9. The audio data is first subjected to the pre-emphasis module 901 for pre-emphasis processing. The pre-emphasis process uses a high-pass filter, which is only allowed to be higher than a certain frequency. The signal components pass through, and the signal components below this frequency are suppressed, thereby removing unnecessary low-frequency interference such as human conversation, footsteps, and mechanical noise in the audio data, and flattening the frequency spectrum of the audio signal. The mathematical expression of the high-pass filter is:

H(z) = 1-az ^-1

Among them, a is the correction coefficient, which generally ranges from 0.95 to 0.97, and z is the audio signal.

The noise-removed audio data is subjected to framing processing by the framing and windowing module 902 to obtain audio data corresponding to different audio frames.

Illustratively, in this embodiment, audio data containing 1024 data points is divided into one frame. When the sampling frequency of the audio data is selected as 16000 Hz, the duration of one frame of audio data is 64 ms. In order to avoid excessive changes between the two frames of data, and to avoid the loss of data at both ends of the audio frame after windowing, the audio data is not directly divided into frames in a back-to-back manner, but after each frame of data is fetched , And then slide 32ms to fetch the next frame of data, that is, the data of two adjacent frames overlap by 32ms.

Since the audio data after framing needs to be subjected to discrete Fourier transform in the subsequent feature extraction, a frame of audio data has no obvious periodicity, that is, the left end of the frame and the right end of the frame are not continuous, and after Fourier transform, it will be compared with the original data. Errors will occur. The more frames are divided, the greater the error will be. Therefore, in order to make the framed audio data continuous and each frame of audio data exhibits the characteristics of a periodic function, it is necessary to perform frame and window processing by the frame and window module 902 .

In a possible implementation manner, a Hamming window is used to perform windowing processing on audio frames. Multiply each frame of data by the Hamming window function, and the resulting audio data has obvious periodicity. The functional form of the Hamming window is:

Where n is an integer, n ranges from 0 to M, and M is the number of Fourier transform points. Illustratively, 1024 data points are taken as Fourier transform points in this embodiment.

In a possible implementation, after the audio data is framed and windowed, it needs to perform feature extraction to obtain a feature matrix that can be recognized by the voice recognition model, that is, extract the Mel-Frequency Cepstral coefficient of the audio frame. Coefficients, MFCC).

Since it is difficult to obtain the signal characteristics of the audio signal from the transformation in the time domain, it is usually necessary to convert the time domain signal into the energy distribution in the frequency domain for processing. Therefore, the terminal first inputs the audio frame data to the Fourier transform module 903 for processing. Fourier transform, and then input the audio frame data after Fourier transform to the energy spectrum calculation module 904 to calculate the energy spectrum of the audio frame data. In order to convert its energy spectrum into a Mel spectrum that conforms to human hearing, the energy spectrum needs to be input to the Mel filter processing module 905 for filtering processing; the mathematical expression of the filtering processing is:

Among them, f is the frequency point after Fourier transform.

After obtaining the Mel spectrum of the audio frame, the terminal uses a discrete cosine transform (Discrete Cosine Transform, DCT) module 906 to log it, and the obtained DCT coefficient is the MFCC feature.

Illustratively, the embodiment of the present application selects 40-dimensional MFCC features. When the terminal actually extracts the features, the length of the audio data input window is selected to be 1056 ms, and the time length of one frame of signal is 64 ms. There is a gap between two adjacent frames of data. 32ms overlap, so each input window data of 1056ms corresponds to a matrix of 32*40 generated features.

2. Input the audio feature matrix into the voice recognition model to obtain the alarm bell recognition result output by the voice recognition model. The alarm bell recognition result is used to indicate whether the ambient sound contains the subway closing alarm bell.

In a possible implementation manner, the terminal inputs the audio feature matrix obtained after feature extraction of the audio frame into the voice recognition model, and the model recognizes whether the current audio frame contains the subway closing alarm bell, and outputs the recognition result.

In a possible implementation, if the terminal cannot obtain the subway closing alarm bell of the current city autonomously, the user needs to collect the subway closing alarm bell in advance. After the subway closing alarm bell is collected, the audio of the subway closing alarm bell will be included. The data is also subjected to the framing processing and feature extraction process in the above step 1, and the audio feature matrix of the subway closing alarm bell is saved locally.

Correspondingly, when identifying whether the ambient sound contains the subway door-closing alarm bell, the audio feature matrix after preprocessing the environmental sound and the audio feature matrix corresponding to the subway door-closing alarm bell can be directly input into the voice recognition model to facilitate the sound The recognition model is used to compare the similarity of the two audio feature matrices to obtain the similarity of the output of the voice recognition model, so as to determine whether the ambient sound contains the subway closing alarm bell according to the similarity.

Schematically, the higher the similarity, the more similar the two audio feature matrices are. Correspondingly, the more likely the ambient sound contains the subway closing alarm bell. Therefore, a similarity threshold is set. When the output of the voice recognition model is similar If the degree is greater than the similarity threshold, it is determined that the ambient sound includes the subway closing alarm bell; conversely, if the similarity is lower than the similarity threshold, it is determined that the ambient sound does not include the subway closing alarm bell.

Optionally, the similarity threshold used to determine whether the ambient sound includes a subway closing alarm bell may be 90%.

Optionally, when training the voice recognition model, you can use the sample audio feature matrix, the standard audio feature matrix, and manually label the similarity value between the sample audio feature matrix and the standard audio feature matrix to train the voice recognition model to make the voice recognition The model has the function of comparing the similarity between two audio data; correspondingly, in the actual application process, the audio feature matrix obtained after the environmental sound preprocessing and the pre-stored audio matrix input containing the subway closing alarm bell can be directly input In the voice recognition model, the alarm bell recognition result output by the voice recognition model is obtained, that is, the similarity of the two audio feature matrices.

3. If the alarm bell recognition result indicates that the ambient sound includes the subway closing alarm bell, it is determined that it is in the subway startup stage.

Because the terminal processes the audio data into frames before recognizing the subway closing alarm bell, and the time of one frame of audio is very short, when an audio frame contains the target alarm bell, it cannot be ruled out that there are other similar sounds or features When the data processing during the extraction produces an error, it cannot be immediately determined that the ambient sound contains the subway closing alarm bell. Therefore, the terminal sets a predetermined time period, and when the output result of the voice recognition model indicates that the number of audio frames containing the subway door-closing alarm bell within the predetermined time period reaches the number threshold, it is determined that the ambient sound includes the subway door-closing alarm bell, that is, it is determined that the subway is started.

Illustratively, the terminal sets the predetermined duration to 5 seconds and the number threshold to 2. When the terminal recognizes that 2 or more audio frames contain the subway closing alarm bell within 5 seconds, it determines that the current ambient sound is Including the subway closing alarm bell, which confirms the subway start.

In the embodiment of the present application, the environmental sound is collected by a microphone, and the environmental sound is preprocessed to obtain an audio matrix, which is input into the voice recognition model, and the output of the alarm bell recognition result is used to determine whether the environmental sound contains the target alarm bell. In this way, it is determined whether the subway is started, the accuracy of determining the start of the subway is improved, and the timing of the first acceleration data collection is accurately controlled.

In a possible application scenario, after the terminal determines the running direction of the subway based on the second acceleration data, it can be compared with the running direction the user expects to take the subway. If the two directions match, it is determined that the user is taking the correct ride. , It is determined that the user has taken the wrong ride, so as to promptly remind the user to transfer, and reduce the time waste caused by taking the wrong subway.

Schematically, as shown in FIG. 10, it shows a flowchart of a method for determining whether a user is taking the subway correctly according to an exemplary embodiment of the present application. In this embodiment, the method is applied to a terminal as an example for description. Methods include:

Step 1001: Determine the end station according to the current station and the target station, and the target station is located between the current station and the end station.

Among them, the current station indicates the station where the subway currently stops, that is, the user starts to take the subway from the current station; the target station indicates the next station to which the subway starts from the current station.

Among them, the method for determining the current station and the target station may be that when the user takes the terminal to take the subway, the terminal collects the prompt sound when the subway starts or arrives, and the prompt sound contains the name of the current station and the station of the next station. name.

For the same subway line, under different subway operating directions, that is, the starting station and the terminal station of the subway are opposite, the next station corresponding to the same current station is not the same. Therefore, in a possible implementation manner, when the user carries the terminal and rides In the subway, the terminal determines the actual terminal station of the subway operation by obtaining the names of the current station and the target station (next station).

Optionally, the terminal can also obtain the user's location information in real time through map applications, and determine the user's current site and target site based on the user's location information and subway route map, thereby determining the terminal station. This does not constitute a limitation.

In step 1002, according to the current station and the terminal station, the target subway running direction is searched from the database. The target subway running direction is the running direction when the subway runs from the current station to the terminal station.

Among them, the database stores subway information of major cities across the country, and the database may include correspondences between cities, subway line identifiers, station names, actual operating directions, and terminal directions.

Schematically, the database may be as shown in Table 1.

Table I

city	Line identification	Site name	Actual running direction	Terminal
City 1	12	Site A	northeast	Site B
To	To	To	southwest	Site C
City 2	1	Site D	southeast	Site E
To	To	To	northwest	Site F

In a possible implementation, when the terminal determines that the current station is station A and the terminal station is station B, then according to the current station and the terminal station, searching from the database (Table 1) shows that the target subway running direction is To the northeast.

In step 1003, if the determined subway running direction does not match the target subway running direction, a transfer prompt is performed.

In a possible implementation, if the subway running direction determined in step 1002 does not match the target subway running direction, it is determined that the user may have an incorrect subway riding direction, that is, a transfer prompt is performed so that the user can get off the train in time , To avoid wasting time caused by taking the wrong subway.

Wherein, the transfer prompt may be a voice reminder, a vibration reminder, etc., and the embodiment of the present application does not limit the manner of the transfer prompt.

Optionally, if the determined subway running direction matches the target subway running direction, it is determined that the user is taking the subway correctly, and a confirmation reminder may also be issued, so that the user can determine that the current subway is taking the correct subway without changing.

It should be noted that this embodiment may be executed after step 406 shown in FIG. 4 or step 203 shown in FIG. 2.

In this embodiment, through the preset database in the terminal, the determined subway operation direction can be compared with the actual subway operation direction searched in the database, so as to promptly remind the user to transfer when the user takes the wrong subway, and avoid taking the wrong subway. The resulting wasted time.

In another possible application scenario, the terminal station of the actual subway operation can be determined according to the determined direction of subway operation and the name of the current station, so as to determine the actual subway line the user is currently taking according to the current station and terminal station. Thereby feedback to the user, so that the user can confirm whether the subway line is consistent with the subway line the user expects to take.

In a possible implementation manner, the process of determining the subway line the user rides on according to the subway running direction may include the following steps:

1. Get the current site.

Among them, the current station is the station where the subway that the user rides currently stops at.

In a possible implementation manner, the terminal may collect the subway opening prompt sound or the subway arrival prompt sound to obtain the current station where the user is located.

Optionally, the current station where the user is located can also be determined by obtaining the user's location information and subway route map.

2. According to the current station and the direction of subway operation, determine the target station to which the subway runs from the current station.

Among them, the target station may be the terminal station of the subway operation currently taken by the user, or the next station reached by the subway operation taken by the user, or any station between the current station and the terminal station in the current subway operation route. Among them, the subway information database of major cities across the country is established in the terminal, and the terminal station or the next station that the subway is going to at the current station can be determined in the database according to the current station and the running direction of the subway. The database can refer to the above embodiment, and this embodiment will not be repeated here.

Although the running direction of the subway determined according to the second acceleration data in the above embodiment may be different from the actual running direction of the subway, it is because for the same subway line where the starting point and the end station of the subway are opposite, the corresponding subway running direction The opposite is true. For example, the subway line runs from station A to station B. When the subway runs from station A to station B, the corresponding subway direction is northeast. When the subway runs from station B to station A, the corresponding The running direction of the subway is the southwest direction. Therefore, in a possible implementation manner, the corresponding running terminal can be found from the database according to the determined running direction of the subway and the current station.

In an illustrative example, if the subway running direction is southeast and the current station is station D, it can be seen from Table 1 that the target station to which the user currently takes the subway is station E.

3. Determine the target subway route for subway operation based on the current station and target station.

In a possible implementation manner, after the target site is determined, the complete subway route corresponding to the subway the user is currently riding on can be determined according to the current site and the target site, and the subway route is fed back to the user for the user to determine Whether the current subway ride is the subway that they expect to ride, if not, the user can transfer the subway in time to avoid the user from increasing the transportation cost of the user's travel due to the mistake of taking the subway.

In this embodiment, it is possible to estimate the target station to which the subway currently traveled by the user is based on the predicted subway running direction in the above embodiment, so as to feedback the current subway running direction to the user, that is, the end of the subway running. This allows the user to determine whether the subway station is consistent with the subway station the user expects to take according to the subway operating station, so as to find out whether the subway is taken incorrectly in time, and avoid the waste of time caused by taking the wrong subway.

Please refer to FIG. 11, which shows a structural block diagram of an apparatus for determining a subway running direction provided by an exemplary embodiment of the present application. The device can be implemented as all or a part of the terminal through software, hardware or a combination of the two. The device includes:

The acquiring module 1101 is configured to acquire the first acceleration data collected by the linear acceleration sensor during the subway acceleration phase, where the first acceleration data includes acceleration data in various directions in the terminal coordinate system;

The data conversion module 1102 is configured to convert the first acceleration data into second acceleration data, and the second acceleration data includes acceleration data in various directions in the world coordinate system;

The first determining module 1103 is configured to determine the running direction of the subway according to the second acceleration data within the target duration.

Optionally, a gravity acceleration sensor and a magnetometer are provided in the terminal;

Optionally, the data conversion module 1102 includes:

An acquiring unit, configured to acquire the gravity acceleration data collected by the gravity acceleration sensor and the magnetic force data collected by the magnetometer, the gravity acceleration data and the magnetic force data being based on the terminal coordinate system;

A first determining unit, configured to determine a conversion matrix according to the gravity acceleration data and the magnetic force data, the conversion matrix being used to convert acceleration data in the terminal coordinate system into acceleration data in the world coordinate system;

The data conversion unit is configured to convert the first acceleration data into the second acceleration data through the conversion matrix.

Optionally, the linear acceleration sensor, the gravity acceleration sensor, and the magnetometer perform data collection according to a target sampling frequency;

Optionally, the obtaining unit is further used for:

Acquiring the nth gravitational acceleration data collected by the gravitational acceleration sensor and the nth magnetic force data collected by the magnetometer, where n is an integer greater than or equal to 2;

Optionally, the first determining unit is further configured to:

If the similarity between the nth gravitational acceleration data and the n-1th gravitational acceleration data is greater than the similarity threshold, and the similarity between the nth magnetic data and the n-1th magnetic data is greater than the similarity threshold, then The n-1th conversion matrix is determined to be the nth conversion matrix, and the n-1th conversion matrix is determined according to the n-1th gravitational acceleration data and the n-1th magnetic force data;

If the similarity between the nth gravitational acceleration data and the n-1th gravitational acceleration data is less than the similarity threshold, and/or the similarity between the nth magnetic force data and the n-1th magnetic force data Less than the similarity threshold, determine the nth conversion matrix according to the nth gravitational acceleration data and the nth magnetic force data;

Optionally, the data conversion unit is further used for:

The first acceleration data is converted into the second acceleration data through the nth conversion matrix.

Optionally, the first determining module 1103 includes:

The first output unit is configured to input the second acceleration data within the target duration into a running direction prediction model to obtain a running direction prediction result output by the running direction prediction model. The running direction prediction result includes each candidate The predicted probability corresponding to the running direction;

The second determining unit is configured to determine the running direction of the subway according to the predicted probability of each candidate running direction.

Optionally, the running direction prediction model adopts a CNN structure, and the running direction prediction model is obtained by training based on sample acceleration data and sample running direction;

The sample acceleration data is the acceleration data of each direction in the world coordinate system during the acceleration phase of the subway, and the sample running direction is the actual running direction of the subway.

Optionally, the device further includes:

A second determining module, configured to determine an end station based on a current station and a target station, where the target station is located between the current station and the end station;

The searching module is configured to search for a target subway running direction from a database according to the current station and the terminal station, where the target subway running direction is the running direction of the subway when the subway runs from the current station to the terminal station;

The prompting module is used to perform transfer prompts if the determined running direction of the subway does not match the target running direction of the subway.

Optionally, the subway acceleration phase includes a subway startup phase;

Optionally, the device further includes:

Collection module, used to collect ambient sound through a microphone;

The third determining module is configured to determine that the subway is in the start-up phase when it is recognized that the ambient sound includes the subway door-closing alarm bell.

Optionally, the third determining module includes:

A feature extraction unit, configured to perform audio feature extraction on the environmental sound to obtain an audio feature matrix;

The second output unit is configured to input the audio feature matrix into a voice recognition model to obtain a warning bell recognition result output by the voice recognition model, and the warning bell recognition result is used to indicate whether the ambient sound contains the subway Door-closing alarm bell;

The third determining unit is configured to determine that it is in the start-up phase of the subway if the alarm bell recognition result indicates that the environmental sound includes the subway door-closing alarm bell.

Optionally, the first determining unit is further configured to:

Determine, according to the gravitational acceleration data and the magnetic force data, the target vector corresponding to the unit vector in each direction in the world coordinate system in the terminal coordinate system;

Determine the conversion matrix between the world coordinate system and the terminal coordinate system according to the unit vector and the target vector.

In the embodiment of the present application, when the subway is in the acceleration phase, the terminal collects acceleration data (first acceleration data) in various directions in the terminal coordinate system through the linear acceleration sensor, and transforms the first acceleration data to obtain the world coordinate system Acceleration data (second acceleration data) in each direction, so as to determine the subway running direction according to the second acceleration data within the target duration. By converting the first acceleration data collected by the terminal into the second acceleration data, due to the uniqueness of the world coordinate system, the influence of the terminal attitude on the acquired acceleration direction is avoided, thereby improving the accuracy of determining the direction of subway operation. In addition, , Through the comprehensive analysis of the second acceleration data within the target duration to determine the direction of the subway, the problem of inaccurate determination of the direction of the subway due to the instability of a single acceleration direction can be avoided.

Please refer to FIG. 12, which shows a structural block diagram of a terminal 1200 according to an exemplary embodiment of the present application. The terminal 1200 may be an electronic device with an application installed and running, such as a smart phone, a tablet computer, an e-book, or a portable personal computer. The terminal 1200 in this application may include one or more of the following components: a processor 1210, a memory 1220, a screen 1230, and a sensor 1240.

The processor 1210 may include one or more processing cores. The processor 1210 uses various interfaces and lines to connect various parts of the entire terminal 1200, and executes the terminal by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1220, and calling data stored in the memory 1220. The various functions and processing data of the 1200. Optionally, the processor 1210 may adopt at least one of digital signal processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA). A kind of hardware form to realize. The processor 1210 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a modem, and the like. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is used for rendering and drawing the content that needs to be displayed on the screen 1230; the modem is used for processing wireless communication. It is understandable that the above-mentioned modem may not be integrated into the processor 1210, but may be implemented by a communication chip alone.

The memory 1220 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 1220 includes a non-transitory computer-readable storage medium. The memory 1220 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 1220 may include a program storage area and a data storage area, where the program storage area may store instructions for implementing the operating system and instructions for implementing at least one function (such as touch function, sound playback function, image playback function, etc.) , The instructions used to implement the foregoing various method embodiments, etc., the operating system may be the Android system (including the system based on the in-depth development of the Android system), the IOS system developed by Apple (including the system based on the in-depth development of the IOS system) Or other systems. The data storage area can also store data created during use of the terminal 1200 (such as phone book, audio and video data, chat record data) and the like.

The screen 1230 may be a capacitive touch display screen, which is used to receive a user's touch operation on or near any suitable object such as a finger, a touch pen, etc., and display the user interface of each application program. The touch screen is usually set on the front panel of the terminal 1200. The touch screen can be designed as a full screen, curved screen or special-shaped screen. The touch display screen can also be designed as a combination of a full screen and a curved screen, or a combination of a special-shaped screen and a curved screen, which is not limited in the embodiments of the present application.

The sensor 1240 is used to collect sensor data. In the embodiment of the present application, the sensor 1240 may include a linear acceleration sensor, a gravity acceleration sensor, and a magnetometer. The linear acceleration sensor is used to collect first acceleration data, and the gravity acceleration sensor is used to obtain gravity acceleration. Data, a magnetometer is used to obtain magnetic data. Optionally, the terminal 1200 may further include a temperature sensor, a humidity sensor, a pressure sensor, a proximity sensor, etc., which are not limited in the embodiment of the present application.

In addition, those skilled in the art can understand that the structure of the terminal 1200 shown in the above drawings does not constitute a limitation on the terminal 1200, and the terminal may include more or less components than those shown in the figure, or a combination of certain components. Components, or different component arrangements. For example, the terminal 1200 also includes radio frequency circuits, photographing components, sensors other than the above-mentioned sensors, audio circuits, wireless fidelity (WiFi) components, power supplies, Bluetooth components and other components, which will not be repeated here.

The embodiments of the present application also provide a computer-readable medium that stores at least one instruction, and the at least one instruction is loaded and executed by the processor to realize the subway running direction as described in each of the above embodiments. The method of determining.

According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the terminal reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the terminal executes the method for determining the subway running direction provided in the various optional implementations of the foregoing aspects.

Those skilled in the art should be aware that, in one or more of the foregoing examples, the functions described in the embodiments of the present application can be implemented by hardware, software, firmware, or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on the computer-readable medium. The computer-readable medium includes a computer storage medium and a communication medium, where the communication medium includes any medium that facilitates the transfer of a computer program from one place to another. The storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

The above are only optional embodiments of this application and are not intended to limit this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in the protection of this application. Within range.

Claims (20)

1. A method for determining the running direction of a subway, the method is used in a terminal, and the method includes:

   During the subway acceleration phase, acquiring first acceleration data collected by the linear acceleration sensor, where the first acceleration data includes acceleration data in various directions in the terminal coordinate system;

   Converting the first acceleration data into second acceleration data, where the second acceleration data includes acceleration data in various directions in the world coordinate system;

   Determine the running direction of the subway according to the second acceleration data within the target duration.
2. The method according to claim 1, wherein a gravity acceleration sensor and a magnetometer are provided in the terminal;

   The converting the first acceleration data into second acceleration data includes:

   Acquiring gravity acceleration data collected by the gravity acceleration sensor and magnetic force data collected by the magnetometer, where the gravity acceleration data and the magnetic force data are based on the terminal coordinate system;

   Determining a conversion matrix according to the gravitational acceleration data and the magnetic force data, the conversion matrix being used to convert acceleration data in the terminal coordinate system into acceleration data in the world coordinate system;

   Through the conversion matrix, the first acceleration data is converted into the second acceleration data.
3. The method according to claim 2, wherein the linear acceleration sensor, the gravitational acceleration sensor, and the magnetometer perform data collection according to a target sampling frequency;

   The acquiring the gravity acceleration data collected by the gravity acceleration sensor and the magnetic force data collected by the magnetometer includes:

   Acquiring the nth gravitational acceleration data collected by the gravitational acceleration sensor and the nth magnetic force data collected by the magnetometer, where n is an integer greater than or equal to 2;

   The determining a conversion matrix according to the gravitational acceleration data and the magnetic force data includes:

   If the similarity between the nth gravitational acceleration data and the n-1th gravitational acceleration data is greater than the similarity threshold, and the similarity between the nth magnetic data and the n-1th magnetic data is greater than the similarity threshold, then The n-1th conversion matrix is determined to be the nth conversion matrix, and the n-1th conversion matrix is determined according to the n-1th gravitational acceleration data and the n-1th magnetic force data;

   If the similarity between the nth gravitational acceleration data and the n-1th gravitational acceleration data is less than the similarity threshold, and/or the similarity between the nth magnetic force data and the n-1th magnetic force data Less than the similarity threshold, determine the nth conversion matrix according to the nth gravitational acceleration data and the nth magnetic force data;

   The converting the first acceleration data into the second acceleration data through the conversion matrix includes:

   The first acceleration data is converted into the second acceleration data through the nth conversion matrix.
4. The method according to any one of claims 1 to 3, wherein the determining the subway running direction according to the second acceleration data within the target duration includes:

   Inputting the second acceleration data within the target duration into a running direction prediction model to obtain a running direction prediction result output by the running direction prediction model, and the running direction prediction result includes the prediction probability corresponding to each candidate running direction;

   The subway running direction is determined according to the predicted probability of each candidate running direction.
5. The method according to claim 4, wherein the running direction prediction model adopts a convolutional neural network CNN structure, and the running direction prediction model is obtained by training according to sample acceleration data and sample running direction;

   The sample acceleration data is the acceleration data of the subway in various directions under the world coordinate system, and the sample running direction is the actual running direction of the subway.
6. The method according to any one of claims 1 to 3, wherein after the determining the subway running direction according to the second acceleration data within the target duration, the method further comprises:

   Determining an end station according to a current station and a target station, where the target station is located between the current station and the end station;

   Searching for a target subway running direction from a database according to the current station and the terminal station, where the target subway running direction is the running direction of the subway when the subway runs from the current station to the terminal station;

   If the determined running direction of the subway does not match the target running direction of the subway, a transfer prompt is performed.
7. The method according to any one of claims 1 to 3, wherein the subway acceleration phase includes a subway startup phase;

   Before acquiring the first acceleration data collected by the linear acceleration sensor during the subway acceleration phase, the method further includes:

   Collect ambient sound through a microphone;

   When it is recognized that the environmental sound includes a subway door-closing alarm bell, it is determined that the subway is in the starting phase.
8. The method according to claim 7, wherein the determining that the subway is in the start-up phase when it is recognized that the ambient sound includes a subway door-closing alarm bell includes:

   Performing audio feature extraction on the ambient sound to obtain an audio feature matrix;

   Inputting the audio feature matrix into a voice recognition model to obtain a warning bell recognition result output by the voice recognition model, the warning bell recognition result being used to indicate whether the ambient sound includes the subway closing warning bell;

   If the alarm bell recognition result indicates that the ambient sound includes the subway closing alarm bell, it is determined that it is in the subway startup stage.
9. The method according to claim 2, wherein said determining a conversion matrix according to said gravitational acceleration data and said magnetic force data comprises:

   Determine, according to the gravitational acceleration data and the magnetic force data, the target vector corresponding to the unit vector in each direction in the world coordinate system in the terminal coordinate system;

   Determine the conversion matrix between the world coordinate system and the terminal coordinate system according to the unit vector and the target vector.
10. A device for determining the running direction of a subway, the device is used in a terminal, and the device includes:

    The acquiring module is configured to acquire the first acceleration data collected by the linear acceleration sensor during the subway acceleration phase, where the first acceleration data includes acceleration data in various directions in the terminal coordinate system;

    A data conversion module, configured to convert the first acceleration data into second acceleration data, the second acceleration data including acceleration data in various directions in the world coordinate system;

    The first determining module is configured to determine the running direction of the subway according to the second acceleration data within the target duration.
11. The device according to claim 10, wherein a gravity acceleration sensor and a magnetometer are provided in the terminal;

    The data conversion module includes:

    An acquiring unit, configured to acquire the gravity acceleration data collected by the gravity acceleration sensor and the magnetic force data collected by the magnetometer, the gravity acceleration data and the magnetic force data being based on the terminal coordinate system;

    A first determining unit, configured to determine a conversion matrix according to the gravity acceleration data and the magnetic force data, the conversion matrix being used to convert acceleration data in the terminal coordinate system into acceleration data in the world coordinate system;

    The data conversion unit is configured to convert the first acceleration data into the second acceleration data through the conversion matrix.
12. The device according to claim 11, wherein the linear acceleration sensor, the gravitational acceleration sensor, and the magnetometer perform data collection according to a target sampling frequency;

    The acquiring unit is also used for:

    Acquiring the nth gravitational acceleration data collected by the gravitational acceleration sensor and the nth magnetic force data collected by the magnetometer, where n is an integer greater than or equal to 2;

    The first determining unit is further configured to:

    If the similarity between the nth gravitational acceleration data and the n-1th gravitational acceleration data is greater than the similarity threshold, and the similarity between the nth magnetic data and the n-1th magnetic data is greater than the similarity threshold, then The n-1th conversion matrix is determined to be the nth conversion matrix, and the n-1th conversion matrix is determined according to the n-1th gravitational acceleration data and the n-1th magnetic force data;

    If the similarity between the nth gravitational acceleration data and the n-1th gravitational acceleration data is less than the similarity threshold, and/or the similarity between the nth magnetic force data and the n-1th magnetic force data Less than the similarity threshold, determine the nth conversion matrix according to the nth gravitational acceleration data and the nth magnetic force data;

    The data conversion unit is also used for:

    The first acceleration data is converted into the second acceleration data through the nth conversion matrix.
13. The device according to any one of claims 10 to 12, wherein the first determining module comprises:

    The first output unit is configured to input the second acceleration data within the target duration into a running direction prediction model to obtain a running direction prediction result output by the running direction prediction model. The running direction prediction result includes each candidate The predicted probability corresponding to the running direction;

    The second determining unit is configured to determine the running direction of the subway according to the predicted probability of each candidate running direction.
14. The device according to claim 13, wherein the running direction prediction model adopts a CNN structure, and the running direction prediction model is obtained by training based on sample acceleration data and sample running direction;

    The sample acceleration data is the acceleration data of the subway in various directions under the world coordinate system, and the sample running direction is the actual running direction of the subway.
15. The device according to any one of claims 10 to 12, wherein the device further comprises:

    A second determining module, configured to determine an end station according to a current station and a target station, the target station being located between the current station and the end station;

    The searching module is configured to search a target subway running direction from a database according to the current station and the terminal station, where the target subway running direction is the running direction of the subway when the subway runs from the current station to the terminal station;

    A prompting module is used to perform transfer prompts if the determined running direction of the subway does not match the target running direction of the subway.
16. The device according to any one of claims 10 to 12, wherein the subway acceleration phase includes a subway startup phase;

    The device also includes:

    Collection module, used to collect ambient sound through a microphone;

    The third determining module is configured to determine that the subway is in the start-up phase when it is recognized that the ambient sound includes the subway door-closing alarm bell.
17. The device according to claim 16, wherein the third determining module comprises:

    A feature extraction unit, configured to perform audio feature extraction on the environmental sound to obtain an audio feature matrix;

    The second output unit is configured to input the audio feature matrix into a voice recognition model to obtain a warning bell recognition result output by the voice recognition model, and the warning bell recognition result is used to indicate whether the ambient sound contains the subway Door-closing alarm bell;

    The third determining unit is configured to determine that it is in the start-up phase of the subway if the alarm bell recognition result indicates that the ambient sound includes the subway door-closing alarm bell.
18. The device according to claim 12, wherein the first determining unit is further configured to:

    Determine, according to the gravitational acceleration data and the magnetic force data, the target vector corresponding to the unit vector in each direction in the world coordinate system in the terminal coordinate system;

    Determine the conversion matrix between the world coordinate system and the terminal coordinate system according to the unit vector and the target vector.
19. A terminal comprising a processor and a memory; the memory stores at least one instruction, and the at least one instruction is used to be executed by the processor to realize the subway operation according to any one of claims 1 to 9 How to determine the direction.
20. A computer-readable storage medium, the storage medium stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the method for determining a subway running direction according to any one of claims 1 to 9.

Images