CN110046670B - Feature vector dimension reduction method and device - Google Patents
Feature vector dimension reduction method and device Download PDFInfo
- Publication number
- CN110046670B CN110046670B CN201910331938.4A CN201910331938A CN110046670B CN 110046670 B CN110046670 B CN 110046670B CN 201910331938 A CN201910331938 A CN 201910331938A CN 110046670 B CN110046670 B CN 110046670B
- Authority
- CN
- China
- Prior art keywords
- feature vector
- dimension
- data
- value
- sample data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application discloses a feature vector dimension reduction method and device. One embodiment of the method comprises: acquiring data to be processed; inputting data to be processed into a target feature extraction network to obtain a first feature vector; and inputting the first feature vector to a pre-trained full-connection layer to obtain a second feature vector, wherein the dimensionality of an output vector of the full-connection layer is the dimensionality of the feature vector after the preset dimensionality reduction. The implementation mode provides a feature vector dimension reduction mechanism based on deep learning, and enriches feature vector dimension reduction methods.
Description
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to a feature vector dimension reduction method and device.
Background
With the development of AI technology, various tasks can be realized through a deep learning model, such as a voice classification task and an image recognition task. In the deep learning model, feature vectors are generally required to be extracted, however, too many feature dimensions extracted in the process of feature extraction often lead to too complicated feature matching and large consumption of system resources, and therefore, a feature vector dimension reduction method is required to be used to reduce the dimensions of the feature vectors.
Disclosure of Invention
The embodiment of the application provides a feature vector dimension reduction method and device.
In a first aspect, some embodiments of the present application provide a feature vector dimension reduction method, including: acquiring data to be processed; inputting data to be processed into a target feature extraction network to obtain a first feature vector; and inputting the first feature vector to a pre-trained full-connection layer to obtain a second feature vector, wherein the dimensionality of an output vector of the full-connection layer is the dimensionality of the feature vector after the preset dimensionality reduction.
In some embodiments, the fully-connected layer comprises a fully-connected layer trained according to the following steps: acquiring a sample data set; performing the following training steps based on the set of sample data: respectively inputting sample data in the sample data set to a target feature extraction network, respectively inputting the output of the target feature extraction network to an initial full-connection layer, and generating a feature vector corresponding to each sample data; determining whether a preset optimization target is reached according to the generated feature vector; in response to determining that the initial fully-connected layer meets the optimization goal, the initial fully-connected layer is treated as a trained fully-connected layer.
In some embodiments, the step of training the fully connected layer further comprises: and responding to the determination that the initial full connection layer does not reach the optimization target, adjusting the network parameters of the initial full connection layer, using the adjusted full connection layer as the initial full connection layer, and continuing to execute the training step.
In some embodiments, before determining whether the preset optimization goal is reached according to the generated feature vector, the step of training to obtain a fully-connected layer further includes: for each dimension of the generated feature vector: counting the maximum value, the minimum value and the median of the generated feature vector on the dimension; for each generated feature vector, comparing the value of the feature vector in the dimension with the counted median in the dimension, updating the value of the feature vector in the dimension to the counted maximum value in the dimension in response to the value of the feature vector in the dimension being greater than the median in the dimension, and updating the value of the feature vector in the dimension to the counted minimum value in the dimension in response to the value of the feature vector in the dimension being less than the median in the dimension.
In some embodiments, determining whether a preset optimization goal is reached based on the generated feature vectors comprises: determining an average value of the generated feature vectors in each dimension; calculating the sum of squares of the differences between the median counted in each dimension and the determined mean; and determining whether the preset optimization target is reached or not according to whether the calculated sum of squares is matched with a preset value or not.
In some embodiments, the sample data in the set of sample data comprises data of the same origin as the data to be processed.
In a second aspect, some embodiments of the present application provide an apparatus for feature vector dimension reduction, the apparatus including: an acquisition unit configured to acquire data to be processed; the characteristic extraction unit is configured to input data to be processed into a target characteristic extraction network to obtain a first characteristic vector; and the dimension reduction unit is configured to input the first feature vector to a pre-trained full-connection layer to obtain a second feature vector, and the dimension of the output vector of the full-connection layer is the dimension of the feature vector after the preset dimension reduction.
In some embodiments, the apparatus further comprises a training unit comprising: an obtaining subunit configured to obtain a sample data set; a training subunit configured to perform the following training steps based on the set of sample data: respectively inputting sample data in the sample data set to a target feature extraction network, respectively inputting the output of the target feature extraction network to an initial full-connection layer, and generating a feature vector corresponding to each sample data; determining whether a preset optimization target is reached according to the generated feature vector; in response to determining that the initial fully-connected layer meets the optimization goal, the initial fully-connected layer is treated as a trained fully-connected layer.
In some embodiments, the training unit further comprises: an adjusting subunit configured to adjust network parameters of the initial fully-connected layer in response to determining that the initial fully-connected layer does not meet the optimization goal, and to continue performing the training step using the adjusted fully-connected layer as the initial fully-connected layer.
In some embodiments, the training subunit is further configured to: for each dimension of the generated feature vector: counting the maximum value, the minimum value and the median of the generated feature vector on the dimension; for each generated feature vector, comparing the value of the feature vector in the dimension with the counted median in the dimension, updating the value of the feature vector in the dimension to the counted maximum value in the dimension in response to the value of the feature vector in the dimension being greater than the median in the dimension, and updating the value of the feature vector in the dimension to the counted minimum value in the dimension in response to the value of the feature vector in the dimension being less than the median in the dimension.
In some embodiments, the training subunit is further configured to: determining an average value of the generated feature vectors in each dimension; calculating the sum of squares of the differences between the median counted in each dimension and the determined mean; and determining whether the preset optimization target is reached or not according to whether the calculated sum of squares is matched with a preset value or not.
In some embodiments, the sample data in the set of sample data comprises data of the same origin as the data to be processed.
In a third aspect, some embodiments of the present application provide an apparatus comprising: one or more processors; a storage device, on which one or more programs are stored, which, when executed by the one or more processors, cause the one or more processors to implement the method as described above in the first aspect.
In a fourth aspect, some embodiments of the present application provide a computer readable medium having stored thereon a computer program which, when executed by a processor, implements the method as described above in the first aspect.
According to the feature vector dimension reduction method and device provided by the embodiment of the application, the data to be processed is obtained; inputting data to be processed into a target feature extraction network to obtain a first feature vector; and inputting the first feature vector to a pre-trained full-connection layer to obtain a second feature vector, wherein the dimension of the output vector of the full-connection layer is the dimension of the feature vector after the dimension reduction is preset, a feature vector dimension reduction mechanism based on deep learning is provided, and feature vector dimension reduction methods are enriched.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is a diagram of an exemplary system architecture to which some of the present application may be applied;
FIG. 2 is a flow diagram of one embodiment of a feature vector dimension reduction method according to the present application;
FIG. 3 is a diagram illustrating an application scenario of the eigenvector dimension reduction method according to the present application;
FIG. 4 is a flow diagram of training of fully connected layers in yet another embodiment of a feature vector dimension reduction method according to the present application;
FIG. 5 is a schematic structural diagram of an embodiment of a feature vector dimension reduction apparatus according to the present application;
FIG. 6 is a block diagram of a computer system suitable for use in implementing a server or terminal of some embodiments of the present application.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Fig. 1 shows an exemplary system architecture 100 to which embodiments of the feature vector dimension reduction method or feature vector dimension reduction apparatus of the present application may be applied.
As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. Various client applications, such as a speech recognition-type application, an image processing-type application, a social-type application, a data processing-type application, an e-commerce-type application, a search-type application, etc., may be installed on the terminal devices 101, 102, 103.
The terminal apparatuses 101, 102, and 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices with display screens, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like. When the terminal apparatuses 101, 102, 103 are software, they can be installed in the electronic apparatuses listed above. It may be implemented as multiple pieces of software or software modules, or as a single piece of software or software module. And is not particularly limited herein.
The server 105 may be a server providing various services, for example, a background server providing support for applications installed on the terminal devices 101, 102, and 103, and the server 105 may obtain to-be-processed data uploaded by the terminal devices 101, 102, and 103; inputting data to be processed into a target feature extraction network to obtain a first feature vector; and inputting the first feature vector to a pre-trained full-connection layer to obtain a second feature vector, wherein the dimensionality of an output vector of the full-connection layer is the dimensionality of the feature vector after the preset dimensionality reduction.
It should be noted that the feature vector dimension reduction method provided in the embodiment of the present application may be executed by the server 105, or may be executed by the terminal devices 101, 102, and 103, and accordingly, the feature vector dimension reduction apparatus may be disposed in the server 105, or may be disposed in the terminal devices 101, 102, and 103.
The server may be hardware or software. When the server is hardware, it may be implemented as a distributed server cluster formed by multiple servers, or may be implemented as a single server. When the server is software, it may be implemented as multiple pieces of software or software modules (e.g., to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
With continued reference to FIG. 2, a flow 200 of one embodiment of a feature vector dimension reduction method according to the present application is shown. The feature vector dimension reduction method comprises the following steps:
In this embodiment, an execution subject (for example, a server or a terminal shown in fig. 1) of the feature vector dimension reduction method may acquire data to be processed by a wired or wireless connection. The data to be processed may be voice, image or text data to be processed.
In this embodiment, the executing entity may input the data to be processed acquired in step 201 to a target feature extraction network to obtain a first feature vector. The target feature extraction network may be any feature extraction network for which the feature to be output is to be subjected to a dimension reduction operation. As an example, the target feature extraction network may include several convolutional layers, may further include a fully connected layer, and the like. The first eigenvector is the vector to be dimension reduced.
And 203, inputting the first feature vector to a pre-trained full-connection layer to obtain a second feature vector, wherein the dimensionality of an output vector of the full-connection layer is the dimensionality of a feature vector after preset dimensionality reduction.
In this embodiment, the executing entity may input the first feature vector obtained in step 202 to a fully-connected layer trained in advance to obtain a second feature vector, where a dimension of an output vector of the fully-connected layer is a dimension of a feature vector after dimension reduction set in advance. The training of the full connection layer can adopt a supervised Learning method or an Unsupervised Learning method, wherein the Unsupervised Learning method (Unsupervised Learning) means that in practical application, a large number of samples which are not labeled or a small number of samples which are labeled exist, and the mutual relation among the samples is learned. When the supervised learning method is adopted for training, the samples and the labeled information in the sample set can be respectively used as input and expected output, and an initial full-connection layer is trained by using a machine learning method. The unsupervised learning method can realize the training of the full connection layer by establishing an optimization target. In some optional implementations of this embodiment, the sample data in the sample data set of the training fully-connected layer includes data that is the same as the source of the data to be processed. The training of the fully-connected layer may end when a preset optimization goal is reached or a preset number of iterations is completed.
Feature dimension reduction (Feature dimension reduction) can also be performed by Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and the like. A deep learning method is introduced, and a full connection layer is added for dimension reduction, so that the problems of complex dimension reduction calculation and poor mobility in the method can be solved, and end-to-end calculation can be completed.
The method provided by the above embodiment of the present application obtains data to be processed; inputting data to be processed into a target feature extraction network to obtain a first feature vector; and inputting the first feature vector to a pre-trained full-connection layer to obtain a second feature vector, wherein the dimension of the output vector of the full-connection layer is the dimension of the feature vector after the dimension reduction is preset, a feature vector dimension reduction mechanism based on deep learning is provided, and feature vector dimension reduction methods are enriched.
With continued reference to fig. 3, fig. 3 is a schematic diagram of an application scenario of the feature vector dimension reduction method according to the present embodiment. In the application scenario of fig. 3, a server 301 obtains data 303 to be processed submitted by a user through a terminal 302, and then inputs the data 303 to be processed into a target feature extraction network 304 to obtain a first feature vector 305; and inputting the first feature vector 305 to a pre-trained fully-connected layer 306 to obtain a second feature vector 307, wherein the dimension of the output vector of the fully-connected layer 306 is the dimension of the feature vector after the dimension reduction is set in advance.
With further reference to FIG. 4, a training flow 400 for a fully connected layer in yet another embodiment of a feature vector dimension reduction method according to the present application is illustrated. The training process 400 includes the following steps:
In this embodiment, an executing subject (e.g., a server or a terminal shown in fig. 1) of the feature vector dimension reduction method may first obtain a sample data set. The steps 402-405 of training the full convolution network may then be performed based on the set of sample data. The sample data set may include sample text, sample voice, sample image, and other data. One sample data set may be a batch, and the batch size (the number of samples in the batch) may be set according to actual needs.
In some optional implementations of this embodiment, the sample data in the sample data set includes data from the same source as the data to be processed. In the implementation mode, the source of the sample data is the same as that of the data to be processed, so that the accuracy of the data processing result can be further improved.
In this embodiment, the executing agent may input sample data in the sample data set acquired in step 401 to the target feature extraction network respectively. Thereby obtaining the feature vector of the dimension to be reduced.
And 403, respectively inputting the output of the target feature extraction network to the initial full-connection layer, and generating a feature vector corresponding to each sample data.
In this embodiment, the executing agent may input the output of the target feature extraction network in step 402 to the initial fully-connected layer, and generate a feature vector corresponding to each sample data. And the dimensionality of the initial fully-connected layer output vector is the preset dimensionality reduced dimensionality. The parameters of the initial fully-connected layer may be empirically determined or adjusted.
And step 404, determining whether a preset optimization target is reached according to the generated feature vector.
In this embodiment, the execution subject may determine whether a preset optimization target is reached according to the feature vector generated in step 403. The optimization objective may include an indicator, such as a recall, that evaluates the output results of the model, or a function value of a loss function.
In some optional implementation manners of this embodiment, before determining whether the preset optimization goal is reached according to the generated feature vector, the step of training to obtain the fully-connected layer further includes: for each dimension of the generated feature vector: counting the maximum value, the minimum value and the median of the generated feature vector on the dimension; for each generated feature vector, comparing the value of the feature vector in the dimension with the counted median in the dimension, updating the value of the feature vector in the dimension to the counted maximum value in the dimension in response to the value of the feature vector in the dimension being greater than the median in the dimension, and updating the value of the feature vector in the dimension to the counted minimum value in the dimension in response to the value of the feature vector in the dimension being less than the median in the dimension.
In the implementation mode, by increasing data distribution, the distance between data and the maximum and minimum characteristic values is reduced by taking the median of partial characteristic statistics as a standard and taking the maximum and minimum characteristic values as a center, so that the expressive space of the data is expanded. As an example, the maximum value of the generated feature vector in a certain dimension is 1, the minimum value is 0, and the median is 0.5, so that the value of the generated feature vector in the dimension is updated to 1 if it is greater than 0.5, is updated to 0 if it is less than 0.5, and is updated to 1 and 0.5 if it is equal to 0.5.
In some optional implementations of the present embodiment, determining whether the preset optimization goal is reached according to the generated feature vector includes: determining an average value of the generated feature vectors in each dimension; calculating the sum of squares of the differences between the median counted in each dimension and the determined mean; and determining whether the preset optimization target is reached or not according to whether the calculated sum of squares is matched with a preset value or not.
As an example, the fully-connected output has n dimensions, if 8 sample data are collected in one sample data set, that is, 8 eigenvectors are generated, it is counted that the median of the generated 8 eigenvectors in a certain dimension is 0.9, the maximum value and the minimum value are 1.2 and 0.6 respectively, and 4 values of the 8 eigenvectors in the dimension smaller than 0.9 and larger than 0.9 are all 4, then the values of the 8 eigenvectors in the dimension after updating are respectively: 0.6, 0.6, 0.6, 0.6, 1.2, 1.2, 1.2, 1.2, the mean value in this dimension is 0.9, and the sum of the squares of the differences between the median counted in this dimension and the determined mean value is 0. By analogy, the mean values in other dimensions can be determined, and finally the sum of the squares of the differences between the median counted in each dimension and the determined mean value can be calculated. The preset value can be set according to actual needs, matching with the preset value can include that the difference value with the preset value is smaller than a certain value, or equal to the preset value, and the preset value can include 0.
In this embodiment, the executing entity may regard the initial fully-connected layer as the fully-connected layer after training in response to determining that the initial fully-connected layer reaches the optimization goal in step 404.
In some optional implementations of this embodiment, the step of training to obtain the fully-connected layer further includes: and responding to the determination that the initial full connection layer does not reach the optimization target, adjusting the network parameters of the initial full connection layer, using the adjusted full connection layer as the initial full connection layer, and continuing to execute the training step. In this implementation, the unused sample data may be used to form a sample data set, and the training steps described above may be continued. The execution subject may adjust the network parameters of the initial fully-connected layer by using a Back propagation Algorithm (BP Algorithm) and a gradient descent method (e.g., a small batch gradient descent Algorithm). It should be noted that the back propagation algorithm and the gradient descent method are well-known technologies that are currently widely researched and applied, and are not described herein again.
As an example, the preset optimization goal is that the sum of the squares of the differences between the counted median and the determined mean value in each dimension is 0. In a certain iteration, the counted median is 0.5, the determined average value is 1, the square sum is not 0, the optimization goal is not reached, the parameter of the fully connected layer is adjusted to enable the counted median to be the same as the determined average value, the square sum of the difference between the counted median and the determined average value is 0, and the optimization is finished.
In this embodiment, the use of the full connectivity layer obtained by training in steps 401 to 405 may refer to steps 201 to 203, which are not described herein again.
As can be seen from fig. 4, the purpose of feature dimension reduction is achieved, retraining of the whole depth model is not required, and convergence time is short, so that the scheme described in this embodiment improves data processing efficiency and saves computing resources.
With further reference to fig. 5, as an implementation of the method shown in the above-mentioned figures, the present application provides an embodiment of a feature vector dimension reduction apparatus, where the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 2, and the apparatus may be specifically applied to various electronic devices.
As shown in fig. 5, the feature vector dimension reduction apparatus 500 of the present embodiment includes: an acquisition unit 501, a feature extraction unit 502 and a dimension reduction unit 503. Wherein the acquisition unit is configured to acquire data to be processed; the characteristic extraction unit is configured to input data to be processed into a target characteristic extraction network to obtain a first characteristic vector; and the dimension reduction unit is configured to input the first feature vector to a pre-trained full-connection layer to obtain a second feature vector, and the dimension of the output vector of the full-connection layer is the dimension of the feature vector after the preset dimension reduction.
In this embodiment, the specific processing of the obtaining unit 501, the feature extracting unit 502, and the dimension reducing unit 503 of the feature vector dimension reducing apparatus 500 may refer to step 201, step 202, and step 203 in the corresponding embodiment of fig. 2.
In some optional implementations of this embodiment, the apparatus further includes a training unit, the training unit including: an obtaining subunit configured to obtain a sample data set; a training subunit configured to perform the following training steps based on the set of sample data: respectively inputting sample data in the sample data set to a target feature extraction network, respectively inputting the output of the target feature extraction network to an initial full-connection layer, and generating a feature vector corresponding to each sample data; determining whether a preset optimization target is reached according to the generated feature vector; in response to determining that the initial fully-connected layer meets the optimization goal, the initial fully-connected layer is treated as a trained fully-connected layer.
In some optional implementations of this embodiment, the training unit further includes: an adjusting subunit configured to adjust network parameters of the initial fully-connected layer in response to determining that the initial fully-connected layer does not meet the optimization goal, and to continue performing the training step using the adjusted fully-connected layer as the initial fully-connected layer.
In some optional implementations of this embodiment, the training subunit is further configured to: for each dimension of the generated feature vector: counting the maximum value, the minimum value and the median of the generated feature vector on the dimension; for each generated feature vector, comparing the value of the feature vector in the dimension with the counted median in the dimension, updating the value of the feature vector in the dimension to the counted maximum value in the dimension in response to the value of the feature vector in the dimension being greater than the median in the dimension, and updating the value of the feature vector in the dimension to the counted minimum value in the dimension in response to the value of the feature vector in the dimension being less than the median in the dimension.
In some optional implementations of this embodiment, the training subunit is further configured to: determining an average value of the generated feature vectors in each dimension; calculating the sum of squares of the differences between the median counted in each dimension and the determined mean; and determining whether the preset optimization target is reached or not according to whether the calculated sum of squares is matched with a preset value or not.
In some optional implementations of this embodiment, the sample data in the sample data set includes data from the same source as the data to be processed.
The device provided by the above embodiment of the present application obtains data to be processed; inputting data to be processed into a target feature extraction network to obtain a first feature vector; and inputting the first feature vector to a pre-trained full-connection layer to obtain a second feature vector, wherein the dimension of the output vector of the full-connection layer is the dimension of the feature vector after the dimension reduction is preset, a feature vector dimension reduction mechanism based on deep learning is provided, and feature vector dimension reduction methods are enriched.
Referring now to FIG. 6, shown is a block diagram of a computer system 600 suitable for use in implementing a server or terminal according to an embodiment of the present application. The server or the terminal shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU)601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components may be connected to the I/O interface 605: an input portion 606 such as a keyboard, mouse, or the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program performs the above-described functions defined in the method of the present application when executed by a Central Processing Unit (CPU) 601. It should be noted that the computer readable medium described herein can be a computer readable signal medium or a computer readable medium or any combination of the two. A computer readable medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the C language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present application may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes an acquisition unit, a feature extraction unit, and a dimension reduction unit. Where the names of these units do not in some cases constitute a limitation of the unit itself, for example, the acquisition unit may also be described as a "unit configured to acquire data to be processed".
As another aspect, the present application also provides a computer-readable medium, which may be contained in the apparatus described in the above embodiments; or may be present separately and not assembled into the device. The computer readable medium carries one or more programs which, when executed by the apparatus, cause the apparatus to: acquiring data to be processed; inputting data to be processed into a target feature extraction network to obtain a first feature vector; and inputting the first feature vector to a pre-trained full-connection layer to obtain a second feature vector, wherein the dimensionality of an output vector of the full-connection layer is the dimensionality of the feature vector after the preset dimensionality reduction.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the invention. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.
Claims (10)
1. A method for feature vector dimension reduction, comprising:
acquiring data to be processed, wherein the data to be processed at least comprises one of the following data: voice data, image data, and text data;
inputting the data to be processed into a target feature extraction network to obtain a first feature vector;
inputting the first feature vector to a pre-trained full-connection layer to obtain a second feature vector, wherein the dimensionality of an output vector of the full-connection layer is the dimensionality of a feature vector after preset dimensionality reduction;
the fully-connected layer comprises a fully-connected layer trained according to the following steps:
acquiring a sample data set;
performing the following training steps based on the set of sample data: respectively inputting sample data in the sample data set to the target feature extraction network, respectively inputting the output of the target feature extraction network to the initial full-connection layer, and generating a feature vector corresponding to each sample data; determining whether a preset optimization target is reached according to the generated feature vector; in response to determining that an initial fully-connected layer meets the optimization goal, treating the initial fully-connected layer as a trained fully-connected layer;
before determining whether a preset optimization target is reached according to the generated feature vector, the step of training to obtain the full connection layer further comprises: for each dimension of the generated feature vector: counting the maximum value, the minimum value and the median of the generated feature vector on the dimension; for each generated feature vector, comparing the value of the feature vector in the dimension with the counted median in the dimension, updating the value of the feature vector in the dimension to the counted maximum value in the dimension in response to the value of the feature vector in the dimension being greater than the median in the dimension, and updating the value of the feature vector in the dimension to the counted minimum value in the dimension in response to the value of the feature vector in the dimension being less than the median in the dimension.
2. The method of claim 1, wherein the step of training the fully connected layer further comprises:
and responding to the determination that the initial full connection layer does not reach the optimization target, adjusting the network parameters of the initial full connection layer, using the adjusted full connection layer as the initial full connection layer, and continuing to execute the training step.
3. The method of claim 2, wherein the determining whether a preset optimization goal is reached based on the generated feature vectors comprises:
determining an average value of the generated feature vectors in each dimension;
calculating the sum of squares of the differences between the median counted in each dimension and the determined mean;
and determining whether the preset optimization target is reached or not according to whether the calculated sum of squares is matched with a preset value or not.
4. The method of claim 1, wherein sample data in the set of sample data comprises the same data as the source of the data to be processed.
5. A feature vector dimension reduction apparatus comprising:
an acquisition unit configured to acquire data to be processed, the data to be processed including at least one of: voice data, image data, and text data;
the feature extraction unit is configured to input the data to be processed into a target feature extraction network to obtain a first feature vector;
the dimensionality reduction unit is configured to input the first feature vector to a pre-trained full-connection layer to obtain a second feature vector, and the dimensionality of an output vector of the full-connection layer is the dimensionality of a feature vector after pre-set dimensionality reduction;
the apparatus further comprises a training unit comprising:
an obtaining subunit configured to obtain a sample data set;
a training subunit configured to perform the following training steps based on the set of sample data: respectively inputting sample data in the sample data set to the target feature extraction network, respectively inputting the output of the target feature extraction network to the initial full-connection layer, and generating a feature vector corresponding to each sample data; determining whether a preset optimization target is reached according to the generated feature vector; in response to determining that an initial fully-connected layer meets the optimization goal, treating the initial fully-connected layer as a trained fully-connected layer;
the training subunit further configured to: for each dimension of the generated feature vector: counting the maximum value, the minimum value and the median of the generated feature vector on the dimension; for each generated feature vector, comparing the value of the feature vector in the dimension with the counted median in the dimension, updating the value of the feature vector in the dimension to the counted maximum value in the dimension in response to the value of the feature vector in the dimension being greater than the median in the dimension, and updating the value of the feature vector in the dimension to the counted minimum value in the dimension in response to the value of the feature vector in the dimension being less than the median in the dimension.
6. The apparatus of claim 5, wherein the training unit further comprises:
an adjustment subunit configured to adjust network parameters of the initial fully-connected layer in response to determining that the initial fully-connected layer does not meet the optimization goal, and continue to perform the training step using the adjusted fully-connected layer as the initial fully-connected layer.
7. The apparatus of claim 6, wherein the training subunit is further configured to:
determining an average value of the generated feature vectors in each dimension;
calculating the sum of squares of the differences between the median counted in each dimension and the determined mean;
and determining whether the preset optimization target is reached or not according to whether the calculated sum of squares is matched with a preset value or not.
8. The apparatus of claim 5, wherein sample data in the set of sample data comprises the same data as the source of the data to be processed.
9. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method recited in any of claims 1-4.
10. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910331938.4A CN110046670B (en) | 2019-04-24 | 2019-04-24 | Feature vector dimension reduction method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910331938.4A CN110046670B (en) | 2019-04-24 | 2019-04-24 | Feature vector dimension reduction method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110046670A CN110046670A (en) | 2019-07-23 |
| CN110046670B true CN110046670B (en) | 2021-04-30 |
Family
ID=67278896
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910331938.4A Active CN110046670B (en) | 2019-04-24 | 2019-04-24 | Feature vector dimension reduction method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110046670B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112907582B (en) * | 2021-03-24 | 2023-09-29 | 中国矿业大学 | Mine-oriented image saliency extraction defogging method and device and face detection |
| CN115098647B (en) * | 2022-08-24 | 2022-11-01 | 中关村科学城城市大脑股份有限公司 | Feature vector generation method and device for text representation and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104991959A (en) * | 2015-07-21 | 2015-10-21 | 北京京东尚科信息技术有限公司 | Method and system for retrieving same or similar image based on content |
| CN106886599A (en) * | 2017-02-28 | 2017-06-23 | 北京京东尚科信息技术有限公司 | Image search method and device |
| CN108256450A (en) * | 2018-01-04 | 2018-07-06 | 天津大学 | A kind of supervised learning method of recognition of face and face verification based on deep learning |
| CN108304788A (en) * | 2018-01-18 | 2018-07-20 | 陕西炬云信息科技有限公司 | Face identification method based on deep neural network |
| CN108388544A (en) * | 2018-02-10 | 2018-08-10 | 桂林电子科技大学 | A kind of picture and text fusion microblog emotional analysis method based on deep learning |
| CN109376777A (en) * | 2018-10-18 | 2019-02-22 | 四川木牛流马智能科技有限公司 | Cervical cancer tissues pathological image analysis method and equipment based on deep learning |
-
2019
- 2019-04-24 CN CN201910331938.4A patent/CN110046670B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104991959A (en) * | 2015-07-21 | 2015-10-21 | 北京京东尚科信息技术有限公司 | Method and system for retrieving same or similar image based on content |
| CN106886599A (en) * | 2017-02-28 | 2017-06-23 | 北京京东尚科信息技术有限公司 | Image search method and device |
| CN108256450A (en) * | 2018-01-04 | 2018-07-06 | 天津大学 | A kind of supervised learning method of recognition of face and face verification based on deep learning |
| CN108304788A (en) * | 2018-01-18 | 2018-07-20 | 陕西炬云信息科技有限公司 | Face identification method based on deep neural network |
| CN108388544A (en) * | 2018-02-10 | 2018-08-10 | 桂林电子科技大学 | A kind of picture and text fusion microblog emotional analysis method based on deep learning |
| CN109376777A (en) * | 2018-10-18 | 2019-02-22 | 四川木牛流马智能科技有限公司 | Cervical cancer tissues pathological image analysis method and equipment based on deep learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110046670A (en) | 2019-07-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11640528B2 (en) | Method, electronic device and computer readable medium for information processing for accelerating neural network training | |
| CN109800732B (en) | Method and device for generating cartoon head portrait generation model | |
| CN108520220B (en) | Model generation method and device | |
| CN108416310B (en) | Method and apparatus for generating information | |
| CN110852438A (en) | Model generation method and device | |
| CN109800730B (en) | Method and device for generating head portrait generation model | |
| WO2020207174A1 (en) | Method and apparatus for generating quantized neural network | |
| CN112650841A (en) | Information processing method and device and electronic equipment | |
| CN109829164B (en) | Method and device for generating text | |
| CN111340221A (en) | Method and device for sampling neural network structure | |
| CN110288625B (en) | Method and apparatus for processing image | |
| CN112434620B (en) | Scene text recognition method, device, equipment and computer readable medium | |
| CN111368973A (en) | Method and apparatus for training a hyper-network | |
| CN110046670B (en) | Feature vector dimension reduction method and device | |
| CN113468344B (en) | Entity relationship extraction method and device, electronic equipment and computer readable medium | |
| CN113408970B (en) | User information generation method and device, electronic equipment and computer readable medium | |
| CN111709784B (en) | Method, apparatus, device and medium for generating user retention time | |
| CN111062995B (en) | Method, apparatus, electronic device and computer readable medium for generating face image | |
| CN112148865B (en) | Information pushing method and device | |
| CN112434619A (en) | Case information extraction method, case information extraction device, case information extraction equipment and computer readable medium | |
| CN111797931A (en) | Image processing method, image processing network training method, device and equipment | |
| CN111353585A (en) | Structure searching method and device of neural network model | |
| CN111310896A (en) | Method and apparatus for training neural networks | |
| CN113128677A (en) | Model generation method and device | |
| CN111753111A (en) | Picture searching method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |