WO2020062392A1 - 信号处理装置、信号处理方法及相关产品 - Google Patents

信号处理装置、信号处理方法及相关产品 Download PDF

Info

Publication number
WO2020062392A1
WO2020062392A1 PCT/CN2018/112528 CN2018112528W WO2020062392A1 WO 2020062392 A1 WO2020062392 A1 WO 2020062392A1 CN 2018112528 W CN2018112528 W CN 2018112528W WO 2020062392 A1 WO2020062392 A1 WO 2020062392A1
Authority
WO
WIPO (PCT)
Prior art keywords
processing
signal
instruction
data
target
Prior art date
Application number
PCT/CN2018/112528
Other languages
English (en)
French (fr)
Inventor
胡帅
陈天石
周聖元
张曦珊
Original Assignee
上海寒武纪信息科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN201811145756.XA external-priority patent/CN110968285A/zh
Priority claimed from CN201811145439.8A external-priority patent/CN110968235B/zh
Priority claimed from CN201811152999.6A external-priority patent/CN110969246A/zh
Application filed by 上海寒武纪信息科技有限公司 filed Critical 上海寒武纪信息科技有限公司
Priority to EP18920196.5A priority Critical patent/EP3859488A4/en
Priority to US16/620,540 priority patent/US11703939B2/en
Publication of WO2020062392A1 publication Critical patent/WO2020062392A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/26—Speech to text systems
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223—Execution procedure of a spoken command

Definitions

  • the present invention relates to the field of data processing, and in particular, to a signal processing device, a signal processing method, and related products.
  • Embodiments of the present invention provide a signal processing device, a signal processing method, and related products, which implement a function of processing data by inputting signals, which can accurately identify a user's intention and bring convenience to the user.
  • an embodiment of the present invention provides a signal processing apparatus, including:
  • Signal collector for collecting signals input by the user
  • An instruction converter configured to convert the signal into a processing instruction and target data according to a target signal instruction conversion model, where the target data is processing data of data to be processed;
  • a processor configured to process the target data according to the processing instruction and a target processing model.
  • an embodiment of the present invention further provides a machine learning computing device.
  • the machine learning computing device includes one or more signal processing devices according to the first aspect, and is configured to obtain a to-be-calculated operation from other processing devices. Enter data and control information, and execute specified machine learning operations, and pass the execution results to other processing devices through I / O interfaces;
  • the machine learning computing device includes a plurality of the signal processing devices
  • the plurality of the signal processing devices may be connected and transmit data through a specific structure
  • a plurality of the signal processing devices are interconnected and transmitted data through a fast external device interconnect bus PCIE bus to support larger-scale machine learning operations; a plurality of the signal processing devices share the same control system or have their own A control system; a plurality of the signal processing devices share a memory or have respective memories; an interconnection manner of the plurality of the signal processing devices is an arbitrary interconnection topology.
  • an embodiment of the present invention further provides a combination processing device, where the combination processing device includes the machine learning computing device according to the second aspect, a universal interconnection interface, and other processing devices;
  • the machine learning computing device interacts with the other processing devices to jointly complete a computing operation designated by a user.
  • an embodiment of the present invention further provides a neural network chip.
  • the machine learning chip includes the machine learning computing device according to the second aspect or the combined processing device according to the third aspect.
  • an embodiment of the present invention further provides an electronic device, where the electronic device includes the chip according to the fourth aspect.
  • an embodiment of the present invention further provides a board card, wherein the board card includes: a storage device, an interface device and a control device, and the neural network chip according to the sixth aspect;
  • the neural network chip is connected to the storage device, the control device, and the interface device, respectively;
  • the storage device is configured to store data
  • the interface device is configured to implement data transmission between the chip and an external device
  • the control device is configured to monitor a state of the chip.
  • an embodiment of the present invention further provides a signal processing method, including:
  • an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute a computer program according to the first embodiment of the present application.
  • an embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute the implementation of the present application.
  • the signal collector collects a signal input by the user; the instruction converter converts the signal into a processing instruction and target data according to the target signal instruction conversion model, and the target data is processing data of the data to be processed.
  • a processor processes the target data according to the processing instruction and a target processing model.
  • FIG. 1 is a schematic structural diagram of a signal processing device according to an embodiment of the present invention.
  • FIG. 2 is a schematic partial structural diagram of another signal processing device according to an embodiment of the present invention.
  • FIG. 3 is a schematic partial structural diagram of another signal processing device according to an embodiment of the present invention.
  • FIG. 4A is a schematic partial structural diagram of another signal processing device according to an embodiment of the present invention.
  • 4B is a schematic diagram of a signal processing demonstration provided by an embodiment of the present invention.
  • FIG. 4C is another schematic diagram of signal processing demonstration provided by an embodiment of the present invention.
  • 5A is a schematic structural diagram of a computing device according to an embodiment of the present application.
  • 5B is a structural diagram of a computing device provided by an embodiment of the present application.
  • 5C is a structural diagram of a computing device according to another embodiment of the present application.
  • 5D is a structural diagram of a main processing circuit provided by an embodiment of the present application.
  • 5E is a structural diagram of another computing device according to an embodiment of the present application.
  • 5F is a schematic structural diagram of a tree module provided by an embodiment of the present application.
  • 5G is a structural diagram of another computing device according to an embodiment of the present application.
  • 5H is a structural diagram of still another computing device according to an embodiment of the present application.
  • FIG. 6A is a structural diagram of a combination processing device provided by an embodiment of the present application.
  • 6B is a schematic structural diagram of a computing device according to an embodiment of the present application.
  • FIG. 7A is a structural diagram of another combination processing device according to an embodiment of the present application.
  • FIG. 7B is a schematic structural diagram of a board card provided by an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
  • FIG. 9 is a schematic flowchart of a signal processing method according to an embodiment of the present invention.
  • an embodiment herein means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the invention.
  • the appearances of this phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they independent or alternative embodiments that are mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
  • FIG. 1 is a schematic structural diagram of a signal processing apparatus according to an embodiment of the present invention.
  • the signal processing apparatus 100 includes:
  • the signal collector 1019 is configured to collect a signal input by a user.
  • the above signals include, but are not limited to, voice signals, image signals, text signals, and induction signals.
  • the signal acquisition device can acquire only one of the signals, and can also support the acquisition of multiple signals.
  • the signal processing device 100 further includes a signal filter.
  • the signal filter performs noise reduction processing on the input signal to reduce interference of unwanted data in the signal.
  • the signal collector may be a voice signal collector, which may include but is not limited to: a voice sensor, a microphone, a pickup, or obtaining other audio capture devices, which is not limited herein; the signal collector may also be an image capture
  • the signal collector may include, but is not limited to, cameras, video cameras, and other image acquisition devices, which are not limited herein.
  • the signal collector may also be an inductive signal collector, which may include, but is not limited to, a touch screen, a touchpad, a body sensor, and ultrasound Sensors, brain wave sensors, and other sensing signal acquisition devices.
  • the signal collector 1019 may also receive an environmental sound signal when receiving a voice signal.
  • the signal filter may perform noise reduction processing on the voice signal according to the ambient sound signal.
  • the ambient sound signal is noise to the speech signal.
  • the signal collector 101 described above may include a pair of microphone arrays, which may be used to collect the voice signals and the ambient sound signals, and implement noise reduction processing.
  • the signal collector 1019 when the signal collector 1019 receives an image signal, only part of the image information is valid information, such as gestures, facial expressions, and motion trends.
  • the above signal filter is used to filter out useless information, and only extract useful information in the image, such as gestures, facial expressions, and action trends.
  • the above-mentioned image collector 101 may include a body sensor, which may be used to extract a person's body sensory motion and orientation after acquiring an image for positioning to obtain useful information.
  • the signal in step 101 when the signal collector 1019 receives a text signal, not all of the text information is valid information.
  • the above text signal filter is used to filter out useless information and extract useful information of the text signal, specifically, for example, extract keywords.
  • the foregoing signal processing apparatus further includes a first storage module. After the signal collector collects the signal, the signal processing device stores the signal into a first storage module.
  • the instruction converter 1029 is configured to convert the signal into a processing instruction and target data according to a target signal instruction conversion model, where the target data is processing data of data to be processed.
  • the processing instructions may be used to implement the following functions: data modification, data editing, data integration, data splicing, data deletion, etc., which are not limited herein.
  • the processing instruction may include at least one of the following: an image processing instruction, a voice processing instruction, a text processing instruction, and the like, which are not limited herein.
  • image processing instructions it can be used to achieve the following functions: change part or all of the color of the image, change the brightness of part or all of the image, change the contrast of part or all of the image, etc., replace the content of the image part with other content, edit the image part Enlarge / reduce all content, edit part or all of the content to blur or mosaic, splice multiple images, integrate multiple images, delete part of the content in the image, etc., without limitation here.
  • the processing instruction can be used to implement the following functions: change part or all of the loudness, frequency, and timbre of the voice data, replace part of the voice data with other content, modify part or all of the voice content, delete part of the voice data, and The remaining content is spliced, and multiple pieces of voice data are integrated, etc., which are not limited here.
  • the text processing instruction can be used to implement the following functions: delete text files, edit text files, create folders, file encryption / decryption, file transfer, file compression, etc., which are not limited here.
  • the data may be at least one of the following: text data, voice data, and image data, which are not limited herein.
  • the processing instructions may be defined as the following three types of processing instructions, specifically: logical processing instructions, precise processing instructions, fuzzy processing instructions, and of course, other processing instructions may be included.
  • the processing instruction may include at least an operation type and an operation area.
  • the processing instruction may further include operation data.
  • Operation types can include at least one of the following: image recognition, image segmentation, image enhancement, image restoration, image denoising, background blurring, image stitching, image integration, speech recognition, speech synthesis, speech noise reduction, speech separation, text screenshot , Text editing, etc. are not limited here.
  • Logic processing instructions which are used to process the given logical operation requirements. Take the logic processing instructions for the image as an example, such as removing the football in the picture. This operation includes the detection and positioning of the football area in the picture. Delete the area of the football and merge the area and the surrounding content.
  • the logic processing instruction may include an operation type, a logic operation area, and logic operation data.
  • Precise processing instructions are used to process the operation requirements of a given precise data, such as a given coordinate range. Take the precise processing instructions for the image as an example. For example, if the center point of the figure is the origin and the radius is 10 pixels Change the color of the interval to 255, 255, 0, and 0 respectively.
  • the precise processing instruction may include operation type, precise operation area, and precise operation data.
  • the fuzzy processing instruction is used to process the operation requirements that give fuzzy data, such as operating on the area near the center point.
  • the fuzzy processing instruction may include an operation type, a fuzzy operation area, and fuzzy operation data.
  • processing instructions include related operation instructions such as data input and output, data judgment.
  • Operation types may be in the form of encoding, which is defined by a certain encoding method, such as a sequential encoding method.
  • Operation types include, but are not limited to, movement, deletion, replacement, modification of parameters (such as discoloration, brightness, contrast), modification of shapes (reduction, enlargement, deformation), etc. Specifically, it may include changing the color of the operation area and changing the operation area. Brightness, change the contrast of the operation area, delete the content of the operation area, replace the content of the operation area with other content, expand / reduce / rotate the content of the operation area, blur / fog / mosaicize the content of the operation area, and synthesize the operation area , Stitching operation area, modeling operation area, etc. are not limited here.
  • the operation region may be an entire image or a partial region in the image. According to a given manner, the operation region may include the foregoing logical region, precise region, blurred region, or other regions.
  • a logical area can refer to an area with a certain logical meaning, including designated items (such as football, desks, puppies, etc.), designated backgrounds (such as sea, mountains, etc.), designated parts (such as faces, skeletal frames, birds, etc.) Wings, etc.) and so on. These logical areas are obtained by offline training using algorithms such as machine learning, and can be customized for users by using online training.
  • Precise area can refer to the area with precise positioning, including precise coordinates and ranges, precise requirements and ranges.
  • a circular area with the center point of the graph as the origin and a radius of 10 pixels; positions of all colors in the graph with RGB values of (255, 0, 0).
  • Blurred areas are areas that are pointed but have a slightly blurred location and range, such as the upper left corner of the image, red areas, etc. The positioning and determination of this part of the content can be made through a selection model obtained by offline training of machine learning algorithms, and intelligently select the range to be processed according to the characteristics of the image to be processed.
  • the operation instruction further includes operation data, including logic operation data, precise operation data, and fuzzy operation data.
  • Logic operation data Data with a certain logical meaning, including specified items (such as football, desk, puppy, etc.), specified background (such as sea, mountains, etc.), specified parts (such as human faces, skeletal frames, and bird wings) and many more.
  • the precise data can include precise numerical parameters or precise addresses to the data. Take logic instruction classes as an example.
  • Blurred data can include fuzzy descriptions, such as "darker” in color, "darker” in brightness, and so on.
  • the signal collector receives the voice signal and performs pre-processing to reduce invalid information such as noise.
  • the pre-processed voice signal is sent to the instruction conversion unit.
  • the voice recognition technology is used to analyze and convert the voice signal.
  • the image processing unit receives the image processing instruction, and uses image recognition technology to locate and divide the image to be processed according to the instruction to obtain the target area of the image to be processed. If operation data is required, it is read from the storage unit The required operation data is sent to the image processing unit.
  • the image processing unit uses the operation data to complete a given operation in the target area of the image to be processed according to the processing instruction, and the operation is completed to output the image.
  • the input data to be processed is a portrait picture of a standing person.
  • the input processing requirement is a voice signal, that is, "change the background and stand by the sea” to receive a picture.
  • the portrait image is used as data to be processed, and then sent to the processor for processing.
  • the signal processor 1019 receives the voice signal through the microphone array and performs noise reduction processing to obtain a valid voice request, that is, "change the background and stand by the sea", and instructs the converter.
  • the target area of the known data is stored in the memory 1049; the target area of the data to be processed and the target area of the known data are obtained from the memory 1049, and the two are intelligently integrated, that is, the portrait is appropriately scaled according to the background, according to The preset environment, that is, a solemn and serious posture and expression, is required to appropriately change the pose and expression of the portrait, and then integrate it into the background image, and perform edge softening, processing based on sunlight and person shadows, etc.
  • the processing process ends and the data can be output; otherwise, the user continues to accept the processing request until the end information of the user is received.
  • the form of the end information is not limited, and may be a voice form, an image form, or a form in which the user obtains the information by clicking the end button.
  • the user can enter a new image as a reference image and enter the voice request, and the signal collector 1019 collects the voice request and sends it
  • the input processor 1039 analyzes the requirements, and obtains the target area of the reference image and the portrait pose of the reference image according to the analysis result.
  • the processor 1039 actively sends a confirmation message to the user, asking whether the user is sure to perform the operation. If it is detected that there is no unreasonable situation or the user decides to perform this operation, then edit the portrait pose in the original processed image according to the reference portrait pose, and then perform processing such as border softening to obtain a new processed image.
  • the user's needs can be combined with a variety of input forms. For example, when the user wishes to move the position of the character at this time, the user is required to select the person as the target area by voice, and then obtain the user gesture through the camera. To move the character left or right. Similarly, until the end message from the user is received, the process ends.
  • the instruction converter 1029 converts the signals into processing instructions and target data according to technologies such as voice recognition technology, natural language processing technology, and image recognition technology
  • the instruction converter 1029 retrieves the information from the first storage module. Acquire the signal in step 101 described above.
  • the instruction converter 1029 includes:
  • the first signal identifier 1021 is configured to convert the signal into text information.
  • the recognizer may include one or more sub-recognizers, such as a voice signal sub-recognizer that converts a voice signal into text information through speech recognition technology, and an image signal sub-recognizer that uses the image recognition technology to convert an image or a group of images The effective signal of the image is converted into text information, etc .;
  • multiple types of signals can also be processed at the same time, such as inputting a voice signal and an image signal at the same time, respectively using a voice signal sub-recognizer and an image signal sub-recognizer to process them. Identify.
  • Image recognition technology that is, the recognition and analysis of input image signals, can include but is not limited to the following algorithms: OTSU algorithm, Roberts operator, Sobel operator, Normalized Cuts algorithm for image segmentation, and Bag of words (BoW) method , Parts and structure method, generative method and discriminative method for target recognition and target detection, and the use of kernel correlation filtering (KDF) algorithm, DSST (Discriminative Scale Space Tracker) algorithm for target tracking. Used to extract the operation expressed by the image signal from one or a group of images.
  • KDF kernel correlation filtering
  • DSST Discriminative Scale Space Tracker
  • the above-mentioned image signal processes a group of images.
  • a camera is used to collect multiple images
  • an image signal sub-recognizer is used to segment the collected image to remove invalid information, that is, to segment the foreground and background of the image, remove the background information to obtain the foreground information, and then determine Relevant information such as the position and size of one or more targets, specifically, if edge detection and segmentation techniques are used, such as: edge detection algorithms: sobel edge detection algorithms, laplace edge detection algorithms, etc., and segmentation algorithms: threshold-based Image segmentation algorithm, edge-based image segmentation algorithm, region-based edge segmentation algorithm, graph theory-based edge segmentation algorithm, energy function-based segmentation algorithm, wavelet transform-based segmentation algorithm, and neural network-based segmentation algorithm.
  • edge detection algorithms sobel edge detection algorithms, laplace edge detection algorithms, etc.
  • segmentation algorithms threshold-based Image segmentation algorithm, edge-based image segmentation algorithm, region-based edge segmentation algorithm, graph theory-based edge segmentation algorithm, energy function-
  • Feature extraction techniques can include corner detection such as SIFT, SURF, and HARRIS. If there are multiple candidate targets, Then determine the priority of each candidate; Screen effective targets and use target tracking technologies such as correlation filtering and deep learning methods, which are not limited here. Track the target's motion trajectory to get the meaning expressed by the target, such as obtaining the basic face and human skeleton structure. There is no change, and when the human hand slides from the left to the right, the effective information is obtained by moving the target area of the image to be processed from the left to the right, and then converting the effective image information into text information.
  • text signal processes a section of text. It is assumed that a piece of text information is obtained by a text information collector, and a text signal sub-recognizer is used to segment the collected text to form a text vector. Then using text feature extraction technology, text feature extraction technology can include at least one of the following: TF-IDF, word frequency method, document frequency method, mutual information, expected cross entropy, secondary information entropy, secondary information entropy, x2 statistics method , Text evidence right, genetic algorithm, principal component analysis method, simulated annealing algorithm, N-Gram algorithm, etc. are not limited here, and valid text information is filtered out. Combined with the offline training text matching model, the effective collected text information is converted into text information that can be sent to the signal text converter.
  • TF-IDF word frequency method
  • document frequency method mutual information
  • expected cross entropy secondary information entropy
  • secondary information entropy secondary information entropy
  • x2 statistics method Text evidence right
  • genetic algorithm principal component analysis method
  • the processes of the above-mentioned multiple types of signals are described. Assuming that two or more signal collectors collect information, assuming voice information and image information, then the voice information is processed by the voice signal sub-recognizer to become the text information corresponding to the voice signal, and the image information is processed by the image signal sub-recognizer. It becomes the preset type information corresponding to the image signal.
  • the preset type information may be at least one of the following: text information, image information, voice information, projection information, vibration information, etc., which is not limited herein.
  • a signal text converter 1022 configured to convert the one or more pieces of text information into the processing instruction through a natural language processing technology and the target signal instruction conversion model;
  • a first data recognizer 1023 is configured to perform area division on the data to be processed according to a granularity of a semantic region in the processing instruction and an image recognition technology to obtain the target data.
  • instruction converter 1029 further includes:
  • the obtaining module 1026 is configured to obtain the granularity of the semantic region in the processing instruction.
  • the signal processing device 100 determines that the target data is a face region based on a voice signal, the semantic region is a face region in the data to be processed, and the signal processing apparatus uses a face as a granularity. Acquiring multiple face regions in the data to be processed; when the target data is a background, the signal processing device divides the data to be processed into a background region and a non-background region; when the target data is a red color region, the signal processing The device divides the data to be processed into regions of different colors according to colors.
  • the above-mentioned instruction converter 1029 further includes an interaction module, which is used to complete the interactive operation between the system and the user.
  • the user can send interactive information to the user, receive user feedback information, and generate corresponding instructions based on the feedback information.
  • the instruction is transmitted to the processor 1039 to perform the corresponding operation.
  • the interactive information may be instruction information for prompting the user to perform the specified operation.
  • the specified operation may be set by the user or defaulted by the system.
  • the feedback information may be input by the user. : Voice input, touch input, body motion input, etc. are not limited here.
  • the user of the interaction module determines whether the target area and the processing instruction meet the preset requirements of the judgment model.
  • the preset requirements can be set by the user or the system defaults.
  • the judgment model is a model obtained through offline training, or a user-defined model. If confirmation is received, execution continues.
  • the method for user confirmation may include at least one of the following: multiple methods such as clicking a button, voice input, and image input are not limited herein.
  • the interaction module is used to sequentially zoom a partial area and / or several partial areas of the image to be processed according to a preset order or according to user habits, which is convenient for more detailed operations.
  • the device detects that the image to be processed is a portrait image
  • the eyebrows, eyes, and mouths are taken in order to enlarge them, which is convenient for users to operate them.
  • the speech recognition technology used in the present invention includes, but is not limited to, models such as Artificial Neural Network (ANN), Hidden Markov Model (HMM), and the like.
  • the speech signal sub-recognizer may process the speech signal according to the speech recognition technology;
  • the natural language processing technology includes, but is not limited to, using statistical machine learning, ANN and the like;
  • the semantic understanding unit may extract semantic information according to the natural language processing technology;
  • the above image recognition technologies include, but are not limited to, using edge detection-based methods, threshold segmentation methods, region growth and watershed algorithms, gray-scale integral projection curve analysis, template matching, deformable templates, Hough transform, Snake operator, and Gabor wavelet transform.
  • algorithms such as elastic map matching technology, active shape model, and active appearance model, the image signal sub-recognizer can divide the data to be processed into different regions according to the image recognition technology.
  • the above-mentioned natural language processing technology enables a computer to understand the input voice or text information.
  • Features are selected using algorithms such as WLLR (Weighted Log Likehood) weighted log-likelihood, WFO (Weighted Frequency and Odds) weighted frequency and likelihood.
  • WLLR Weighted Log Likehood
  • WFO Weighted Frequency and Odds
  • semantic understanding technology as an example, the semantics of the input signal are analyzed using semantic tree, topic model, word vector and other technologies to obtain the topics and operations expressed by the input signal, which is convenient for subsequent selection of a suitable model for processing.
  • the example is that the input data to be processed is a voice, the reference data is a text, and the input processing requirement is a voice, and the text is required to be read out and inserted in the middle of the voice to be processed.
  • the signal collector 1019 receives a segment of speech, and performs pre-processing such as noise reduction as data to be processed, and sends it to the memory 1049 for temporary storage.
  • the signal collector 1019 receives the voice signal received through the microphone array and performs noise reduction processing to obtain a valid voice request, and sends it to the processor 1039.
  • the processor 1039 uses speech recognition technology to directly convert the speech signal into the data processing instruction and the target area for the incoming speech signal.
  • the text-to-speech technology is used to convert the input text form reference data. It becomes the reference voice, and then reads the data to be processed from the memory 1049, and processes the reference voice according to the voice characteristics of the voice of the data to be processed, such as the voice intonation. Finally, the reference speech is inserted into the target area in the speech to be processed and smoothed to obtain the processed speech data. If the end message sent by the user is received, the processing process ends and the data is sufficient, otherwise the user continues to accept the processing request until the end message of the user is received.
  • the form of the end information there is no limitation on the form of the end information, and there is no limitation on the form of the end information, which can be a voice form, an image form, or a form in which the user can obtain the information by clicking the end button.
  • the input data to be processed is a piece of Chinese text
  • the input processing requirement is a speech
  • the Chinese text is required to be translated into a piece of English speech suitable for formal occasions.
  • the instruction conversion module 1029 parses the voice processing requirements into an operation instruction and a target area.
  • the processor 1039 first translates the Chinese text into English words with formal words and formal grammar according to the input voice requirements, and then uses the text-to-speech technology to call the processed voice database in the memory 1049 to convert the English text into a relatively Formal voice intonation speech data. If the end message sent by the user is received, the processing process ends and the data is sufficient; otherwise, the user continues to accept the processing request until the end message of the user is received.
  • the form of the end information there is no limitation on the form of the end information, and there is no limitation on the form of the end information, which can be a voice form, an image form, or a form in which the user can obtain the information by clicking the end button.
  • the first signal recognizer 1021 converts the signal into valid text information through the voice recognition technology, image recognition technology, natural language processing, and the like, and saves the text information to the first Storage module.
  • the signal text converter 1022 obtains the text information from the first storage module, and converts the text information into a processing instruction through a natural language processing technology and the target signal instruction conversion model, and saves the processing instruction to the first In the storage module;
  • the first data identifier 1023 divides the data to be processed according to the granularity of the semantic region and the image recognition technology in the processing instruction to obtain the target data, and stores the division result and the target data to In the second storage module.
  • the instruction converter 1029 may also include:
  • a second signal recognizer 1025 configured to directly convert the signal into the processing instruction according to the speech recognition technology, the natural language processing technology, and the target signal instruction conversion model, and save the processing instruction in the first storage module;
  • the second signal recognizer 1025 includes one or more sub-recognizers, such as a voice signal sub-recognizer that recognizes a voice signal by using a voice recognition technology, and an image signal sub-recognizer that recognizes one image or multiple images by using an image recognition technology The effective signal of the image.
  • a voice signal sub-recognizer that recognizes a voice signal by using a voice recognition technology
  • an image signal sub-recognizer that recognizes one image or multiple images by using an image recognition technology The effective signal of the image.
  • the above-mentioned image signal processes a group of images. Specifically, it is assumed that a camera is used to collect multiple images, and an image signal sub-recognizer is used to segment the collected image to remove invalid information, that is, to segment the foreground and background of the image, remove the background information, and obtain the foreground information. Relevant information such as the position and size of one or more targets, specifically, if the use of edge detection technology is used to determine the position and range of the human face, human hand, and human bone structure; second, identify the target, if there are multiple backups Select the target, then determine the priority of each candidate target.
  • the architecture is basically unchanged, and the human hand slides from the left to the right.
  • the valid information is to move the target data of the data to be processed from the left to the right, and then convert the valid image information into text information.
  • the above text signal processes a section of text. It is assumed that a piece of text information is obtained by a text information collector, and a text signal sub-recognizer is used to segment the collected text to form a text vector. Then use text feature extraction technology to filter out valid text information. Combined with the offline training text matching model, the effective collected text information is converted into text information that can be sent to the signal text converter.
  • the processes of the above-mentioned multiple types of signals are described. Assuming that two or more signal collectors collect information, assuming voice information and image information, then the voice information is processed by the voice signal sub-recognizer to become the text information corresponding to the voice signal, and the image information is processed by the image signal sub-recognizer. It becomes text information corresponding to an image signal.
  • the second data identifier 1025 divides the data to be processed according to the granularity of the semantic region, and obtains target data, where the target data is the data to be processed.
  • the data processing area, and the divided result and the above target data are stored in the second storage module.
  • the instruction converter 1029 performs adaptive training on the signal instruction conversion model to obtain the target signal instruction conversion model.
  • the adaptive training of the signal instruction conversion model is performed offline or online.
  • the aforementioned adaptive training of the signal instruction conversion model is performed offline.
  • the instruction converter 1029 performs adaptive training on the signal instruction conversion model based on its hardware to obtain the target signal instruction conversion model;
  • the adaptive training of the signal instruction conversion model is performed online.
  • a cloud server different from the signal converter 102 performs adaptive training on the signal instruction conversion model to obtain a target signal instruction conversion model.
  • the instruction converter 1029 obtains the target signal instruction conversion model from the cloud server.
  • the adaptive training of the signal instruction conversion model described above is supervised or supervised.
  • the above-mentioned adaptive training on the signal instruction conversion model is supervised specifically as follows:
  • the instruction converter 1029 replaces the signal with a prediction instruction according to a signal instruction conversion model; then determines the correlation coefficient between the prediction instruction and its corresponding instruction set, which is a set of instructions obtained manually based on the signal; the instruction converter 1029
  • the signal instruction conversion model is optimized according to the correlation coefficient between the predicted instruction and its corresponding instruction set to obtain the target signal instruction conversion model.
  • the signal processing apparatus 100 further includes:
  • a trainer 1059 configured to replace the signal with a prediction instruction according to the signal instruction conversion model; determine a correlation coefficient between the prediction instruction and its corresponding instruction set; and optimize based on the correlation coefficient between the prediction instruction and its corresponding instruction set The signal instruction conversion model to obtain the target signal instruction conversion model.
  • the above-mentioned adaptive training of the signal instruction conversion model is supervised specifically including: the instruction converter 1029 or the trainer 1059 receives a voice signal containing related commands, such as changing the color of an image, rotating a picture, and the like. Each command corresponds to a set of instructions.
  • the corresponding instruction set is known.
  • the above-mentioned instruction converter 1029 or trainer 1059 uses these speech signals as the input data of the signal instruction conversion model, and obtains the output prediction. instruction.
  • the instruction converter 1029 or the trainer 1059 calculates a correlation coefficient between the prediction instruction and its corresponding instruction set, and adaptively updates parameters (such as weights, offsets, etc.) in the signal instruction conversion model according to the instruction, so as to improve The performance of the signal instruction conversion model described above, and further obtain the target signal instruction conversion model.
  • parameters such as weights, offsets, etc.
  • the instruction converter 1029 or the trainer 1059 receives an image signal containing related commands, such as specifying an object in the image through a gesture, and then requesting it to be moved to the right. Each command corresponds to a set of instructions. For the input image signals used for adaptive training, the corresponding instruction set is known. The above-mentioned instruction converter 1029 or trainer 1059 uses these image signals as the input data of the signal instruction conversion model, and obtains the output prediction. instruction.
  • the instruction converter 1029 or the trainer 1059 calculates a correlation coefficient between the prediction instruction and its corresponding instruction set, and adaptively updates parameters (such as weights, offsets, etc.) in the signal instruction conversion model according to the instruction, so as to improve The performance of the signal instruction conversion model described above, and further obtain the target signal instruction conversion model.
  • parameters such as weights, offsets, etc.
  • the signal processing apparatus 100 further includes:
  • the memory 1049 is configured to store the text information or the processing instruction or the target data.
  • the memory 1049 and the first storage module and the second storage module may be the same storage module, or may be different storage modules.
  • the processor 1039 is configured to process the data to be processed according to the processing instruction and a target processing model.
  • the target data is processing data of the data to be processed
  • the processor 1039 is specifically configured to:
  • the target data is processing data of the data to be processed
  • the processor 1039 is specifically configured to:
  • the processor 1039 is configured to process the image to be processed according to the image processing instruction and the target image processing model to obtain a processed image, and convert the processed image into output information.
  • the processing instruction may be an image processing instruction
  • the target processing model may be a target image processing model
  • the processor 1039 is specifically configured to:
  • the conversion instruction includes a specified data type
  • the specified data type includes one of the following types: a speech type, an image type, and a text type
  • voice information is output.
  • the above-mentioned specified data type may include, but is not limited to, the following types: voice type, image type, and text type, which are not limited herein.
  • the specified data type may be set by the user or defaulted by the system.
  • the data type is specified as the image type, the processed image is directly output.
  • the data type is specified as the text type, the text information is output.
  • the speech type the speech information is output.
  • the processor 1039 includes:
  • the instruction fetching module 1031 is configured to obtain M processing instructions from the storage module within a preset time window, where M is an integer greater than 1;
  • a processing module 1032 is configured to process the target data according to the M processing instructions and the target processing model.
  • processing module 1032 is configured to:
  • the foregoing preset time window can be understood as a preset duration.
  • the processing module 1032 After the obtaining unit 1031 obtains M processing instructions from the storage module 104 within a preset time period, the processing module 1032 performs a pairwise comparison on the M processing instructions, and deletes the instructions with the same function from the M processing instructions. Get N processing instructions. The processing module 1032 processes the data to be processed according to the N processing instructions and the target processing model.
  • the processing module 1032 performs pairwise comparison on the M processing instructions.
  • the processing module 1032 deletes the most expensive one of the processing instructions A and B.
  • the processing module 1032 obtains the processing instructions A and The similarity coefficient of the processing instruction B described above.
  • the similarity coefficient is greater than the similarity threshold, it is determined that the processing instruction A and the processing instruction B have the same function, and the processing module 1032 deletes the most expensive one of the processing instructions A and B; when the similarity coefficient is less than the similarity threshold, the above
  • the processing module 1032 determines that the functions of the processing instructions A and B are different.
  • the processing instructions A and B are any two of the above-mentioned M processing instructions.
  • the processor 1039 can process the data to be processed by including but not limited to ANN and traditional computer vision methods, including but not limited to: body (for example, beautiful legs, breast augmentation), face change, beautify face, change object (cat change Dogs, zebras become horses, apples change to oranges, etc.), backgrounds change (the forest in the back is changed to fields), to obscure (for example, the face covers one eye, and the eyes are reconstructed again), the style is changed (one second change Van Gogh painting style), posture conversion (such as standing and sitting, front face becomes side face), non-oil painting becomes oil painting, changing the background color of the image and changing the seasonal background of the objects in the image, image composition, image fusion , Remove objects or shadows, picture coloring, picture compression, picture vectorization, etc.
  • body for example, beautiful legs, breast augmentation
  • face change beautify face
  • change object cat change Dogs, zebras become horses, apples change to oranges, etc.
  • backgrounds change the forest in the back is changed to fields
  • obscure for
  • the images are processed one or more at a time in sequence, for example, 6 images are stitched, and 2 are stitched each time. Mark with images 1 to 6, respectively,
  • the signal acquisition unit receives the voice signal and performs preprocessing to reduce invalid information, such as noise;
  • the pre-processed voice signal is sent to an instruction conversion unit, which uses speech recognition technology to analyze the voice signal and convert it into a predefined image processing instruction;
  • the image processing unit receives image processing instructions, and uses image recognition technology to locate and divide one or more images to be processed according to the instructions to obtain the corresponding target areas to be processed;
  • the image processing unit uses the operation data to complete a given operation in the target area of the image to be processed;
  • steps A3 to A6 can be repeated. Until all the images are processed, the processed results of each time are read in, and the processed results of each time are processed according to the image processing instructions to obtain the final processing result and output the image.
  • the signal acquisition unit receives voice signals and performs preprocessing to reduce invalid information, such as noise;
  • the pre-processed voice signal is sent to an instruction conversion unit, which uses speech recognition technology to analyze the voice signal and convert it into a predefined image processing instruction;
  • the image processing unit receives image processing instructions, and uses image recognition technology to locate and divide one or more images to be processed according to the instructions to obtain the corresponding target areas to be processed;
  • the image processing unit uses the operation data to complete a given operation in the target area of the image to be processed;
  • the images can be processed according to the image processing model to obtain the required text information and output, thereby achieving the following functions, such as image classification, image recognition, image segmentation, Functions such as image description are not limited here.
  • the image processing model can be used for one or more functions such as image classification, image recognition, image segmentation, or image description.
  • input images and output classification information as follows:
  • the signal acquisition unit receives voice signals and performs preprocessing to reduce invalid information such as noise;
  • the pre-processed voice signal is sent to an instruction conversion unit, and the voice signal is parsed by using speech recognition technology and converted into a predefined image processing instruction;
  • the image processing unit receives the image processing instruction, reads a pre-stored image processing model from the storage unit, and uses the image processing model to process the input image to obtain a classification result;
  • the input image and output image description are as follows:
  • the signal acquisition unit receives the voice signal and performs preprocessing to reduce invalid information such as noise.
  • the pre-processed voice signal is sent to an instruction conversion unit, which uses speech recognition technology to analyze the voice signal and convert it into a predefined image processing instruction;
  • the image processing unit receives the image processing instruction, reads a pre-stored image processing model from the storage unit, divides the input image, extracts one or more target regions, and processes the target regions to obtain a complete text description.
  • D4 utilizes the need to output a voice description, and then uses the voice conversion technology to convert the text description into a voice and output; otherwise, it can directly output. The operation is complete.
  • the input may not be voice information
  • the output is image information, such as text / voice messages, or text / voice messages and image messages to convert text / voice messages into new images. Or add or modify the content of the text / voice message to the original image message.
  • the input text message or the text message converted from the voice message may be pre-processed first to remove noise, obvious errors, etc., and may improve the subsequent processing accuracy.
  • voice message Enter a voice message, and use voice recognition technology to convert the voice message into a text message based on image processing instructions (if you enter a text message, follow-up processing can be performed directly), and the text message follows the specified or default font size, background color, and pattern, etc. Processing is required, and it is added to the designated position of the original image, and the output is sufficient, such as the barrage function.
  • the input text message and the text message just converted from the voice message can be pre-processed first to remove noise, obvious errors, etc., and can improve the subsequent processing accuracy.
  • the split text message is processed separately and its content is converted into corresponding images. For example, if an image is originally input, it is added or modified in the original image, otherwise a new image is generated. Then, according to the transformed generated image, stitching and integration is performed to obtain the image content corresponding to the entire paragraph of text, which can be output. If the input is a voice message, the voice message can be converted into a text message through voice recognition technology, or the voice message can be directly analyzed and analyzed to obtain corresponding image information.
  • the processor 1039 performs adaptive training on the processing model to obtain the target processing model.
  • the adaptive training of the processing model is performed offline or online.
  • the above-mentioned adaptive training of the processing model is performed offline.
  • the processor 1039 performs adaptive training on the processing model based on its hardware to obtain a target signal instruction conversion model.
  • the adaptive training is performed online.
  • a cloud server different from the processor 1039 described above performs adaptive training on the processing model to obtain a target processing model.
  • the processor 1039 obtains the target processing model from the cloud server.
  • the adaptive training of the processing model is supervised or supervised.
  • the above-mentioned adaptive training on the processing model is supervised specifically as follows:
  • the processor 1039 replaces the speech signal with prediction data according to the processing model; then determines the correlation coefficient between the prediction data and its corresponding training data, which is obtained by processing the data to be processed manually based on the speech signal.
  • the processor 1039 optimizes the processing model according to a correlation coefficient between the prediction data and its corresponding training data to obtain the target processing model.
  • the processor 1039 replaces the image signal with prediction data according to the processing model; then determines the correlation coefficient between the prediction data and its corresponding training data, which is obtained by processing the data to be processed manually based on the speech signal
  • the processor 1039 optimizes the processing model according to a correlation coefficient between the prediction data and its corresponding training data to obtain the target processing model.
  • the signal processing apparatus 100 further includes:
  • a trainer 1059 configured to replace the signal with a prediction instruction according to the signal instruction conversion model; determine a correlation coefficient between the prediction instruction and its corresponding instruction set; and optimize based on the correlation coefficient between the prediction instruction and its corresponding instruction set The signal instruction conversion model to obtain the target signal instruction conversion model.
  • the above-mentioned adaptive training of the processing model is supervised.
  • the processor 1039 or the trainer 1059 receives a voice signal containing related commands, such as changing the color of an image, rotating a picture, etc . Each command corresponds to a piece of training data.
  • the corresponding training data is known.
  • the processor 1039 or the trainer 1059 uses these voice signals as input data of the processing model to obtain output prediction data.
  • the processor 1039 or the trainer 1059 calculates a correlation coefficient between the prediction data and the corresponding training data, and adaptively updates parameters (such as weights, offsets, and the like) in the processing model according to the method to improve the processing model. Performance to obtain the above-mentioned target processing model.
  • the processor 1039 or the trainer 1059 receives an image signal containing related commands, such as changing the color of an image, rotating a picture, and the like. Each command corresponds to a piece of training data.
  • the corresponding training data is known.
  • the processor 1039 or the trainer 1059 uses these image signals as input data of the processing model to obtain output prediction data.
  • the processor 1039 or the trainer 1059 calculates a correlation coefficient between the prediction data and the corresponding training data, and adaptively updates parameters (such as weights, offsets, and the like) in the processing model according to the method to improve the processing model. Performance to obtain the above-mentioned target processing model.
  • the instruction converter 1029 of the signal processing device 100 may be used to adaptively train the signal instruction conversion model in the instruction converter 1029 to obtain a target signal instruction conversion model: the signal
  • the processor 1039 of the processing device 100 may be used to adaptively train the processing model in the processor 1039 to obtain a target processing model.
  • the signal processing apparatus 100 further includes:
  • the trainer 105 is configured to adaptively train the signal instruction conversion model in the instruction converter 1029 and the processing model in the processor 1039 to obtain the target signal instruction conversion model and the target processing model.
  • the above-mentioned trainer 1059 can adjust the structure and parameters in the signal instruction conversion model or the processing model through a supervised method or an unsupervised method to improve the performance of the signal instruction conversion model or the processing model, and finally obtain the target signal instruction. Conversion model or target processing model.
  • the signal processing apparatus 100 is presented in the form of a module.
  • the "module” herein may refer to an application-specific integrated circuit (ASIC), a processor and a memory executing one or more software or firmware programs, an integrated logic circuit, and / or other devices that can provide the above functions .
  • ASIC application-specific integrated circuit
  • the above signal collector 1019, instruction converter 1029, processor 1039, storage module 104, and trainer 1059 can be implemented by the devices shown in FIGS. 5A to 8.
  • the instruction converter 1029 of the signal processing device 100 or the processing module 1032 of the processor 1039 is an artificial neural network chip, that is, the instruction converter 1029 and the processing module 1032 of the processor 1039 may be two independent artificial
  • the structure of the neural network chip is shown in Figs. 5A to 8 respectively.
  • the instruction converter 1029 and the processor 1039 can be executed serially or in a soft-flow manner. That is, when the processor 1039 processes the previous image, the instruction converter 1029 can process the next image. Can increase the throughput of hardware and improve the efficiency of image processing.
  • a computing device for performing machine learning calculations.
  • the computing device includes: a controller unit 11 and an operation unit 12, wherein the controller unit 11 is connected to the operation unit 12, and the operation unit 12 includes: a master processing circuit and multiple slave processing circuits;
  • the controller unit 11 is configured to obtain input data and calculation instructions.
  • the method of obtaining input data and calculation instructions may be obtained through a data input and output unit.
  • the data input and output unit may be one or Multiple data I / O interfaces or I / O pins.
  • the above calculation instructions include, but are not limited to, forward operation instructions or backward training instructions, or other neural network operation instructions, such as convolution operation instructions.
  • the specific implementation manner of this application does not limit the specific expressions of the above calculation instructions.
  • the controller unit 11 is further configured to parse the calculation instruction to obtain a plurality of operation instructions, and send the plurality of operation instructions and the input data to the main processing circuit;
  • a main processing circuit 101 configured to perform pre-processing on the input data and transmit data and operation instructions with the plurality of slave processing circuits;
  • a plurality of slave processing circuits 102 configured to perform intermediate operations in parallel according to data transmitted from the master processing circuit and operation instructions to obtain a plurality of intermediate results, and transmit the plurality of intermediate results to the master processing circuit;
  • the main processing circuit 101 is configured to perform subsequent processing on the multiple intermediate results to obtain a calculation result of the calculation instruction.
  • the technical solution provided in this application sets the operation unit into a master-slave structure.
  • it can split the data according to forward-direction calculation instructions, so that multiple slave processing circuits can The part with a large amount of calculation is performed in parallel, thereby increasing the operation speed, saving operation time, and further reducing power consumption.
  • the above machine learning calculation may specifically include an artificial neural network operation
  • the above input data may specifically include input neuron data and weight data.
  • the above calculation result may specifically be: a result of an artificial neural network operation, that is, output neuron data.
  • the operation in the neural network it can be the operation of one layer in the neural network.
  • the realization process is that in the forward operation, after the execution of the artificial neural network in the previous layer is completed, the operation in the next layer is completed.
  • the instruction will calculate the output neuron calculated in the arithmetic unit as the input neuron of the next layer (or perform some operations on the output neuron and use it as the input neuron of the next layer), and at the same time, the weight It is also replaced by the weight of the next layer; in the reverse operation, when the reverse operation of the artificial neural network of the previous layer is completed, the next layer of operation instructions will use the input neuron gradient calculated in the operation unit as the lower
  • the output neuron gradient of one layer is calculated (or some operation is performed on the input neuron gradient and then used as the output neuron gradient of the next layer), and the weight is replaced with the weight of the next layer.
  • the above machine learning calculations may also include support vector machine operations, k-nearest neighbor (k-nn) operations, k-means (k-means) operations, principal component analysis operations, and so on.
  • k-nn k-nearest neighbor
  • k-means k-means
  • principal component analysis operations and so on.
  • the artificial neural network operation is taken as an example to explain the specific scheme of machine learning calculation.
  • the input neuron and output neuron of the multi-layer operation do not refer to the neurons in the input layer and the output layer of the entire neural network, but to the For any two adjacent layers in the network, the neuron in the lower layer of the network forward operation is the input neuron, and the neuron in the upper layer of the network forward operation is the output neuron.
  • the above computing device may further include: the storage unit 10 and the direct memory access unit 50.
  • the storage unit 10 may include one or any combination of a register and a cache.
  • the cache is configured to store the A calculation instruction;
  • the register is used to store the input data and a scalar;
  • the cache is a high-speed temporary cache.
  • the direct memory access unit 50 is used to read or store data from the storage unit 10.
  • the controller unit includes: an instruction storage unit 110, an instruction processing unit 111, and a storage queue unit 113;
  • An instruction storage unit 110 configured to store calculation instructions associated with the artificial neural network operation
  • the instruction processing unit 111 is configured to parse the calculation instruction to obtain multiple operation instructions
  • the storage queue unit 113 is configured to store an instruction queue, where the instruction queue includes a plurality of operation instructions or calculation instructions to be executed according to a sequence of the queue.
  • the main operation processing circuit may also include a controller unit, and the controller unit may include a main instruction processing unit, which is specifically configured to decode instructions into micro instructions.
  • the slave operation processing circuit may also include another controller unit, and the other controller unit includes a slave instruction processing unit, which is specifically configured to receive and process micro instructions.
  • the above micro-instruction may be an instruction next to the instruction.
  • the micro-instruction may be obtained by splitting or decoding the instruction, and may be further decoded into a control signal of each component, each unit, or each processing circuit.
  • the structure of the calculation instruction can be shown in the following table.
  • An ellipsis in the table above indicates that multiple registers or immediate values can be included.
  • the calculation instruction may include: one or more operation domains and an operation code.
  • the calculation instruction may include a neural network operation instruction. Taking a neural network operation instruction as an example, as shown in Table 1, among which, register number 0, register number 1, register number 2, register number 3, and register number 4 may be operation domains. Each of the register number 0, the register number 1, the register number 2, the register number 3, and the register number 4 may be the number of one or more registers.
  • the above register may be off-chip memory. Of course, in practical applications, it may also be on-chip memory for storing data.
  • controller unit may further include:
  • the dependency relationship processing unit 108 is configured to determine whether there is an association relationship between a first operation instruction and a zeroth operation instruction before the first operation instruction when there are multiple operation instructions, such as the first operation instruction and all If the zeroth operation instruction is related, the first operation instruction is cached in the instruction storage unit, and after the zeroth operation instruction is executed, the first operation instruction is extracted from the instruction storage unit. Transmitted to the arithmetic unit;
  • the determining whether there is an association between the first operation instruction and a zeroth operation instruction before the first operation instruction includes:
  • the operation unit 12 may include a master processing circuit 101 and a plurality of slave processing circuits 102.
  • a plurality of slave processing circuits are distributed in an array; each slave processing circuit is connected to an adjacent other slave processing circuit, and a master processing circuit is connected to k in the plurality of slave processing circuits.
  • K slave processing circuits n slave processing circuits in the first row, n slave processing circuits in the m row, and m slave processing circuits in the first column. It should be noted that, The K slave processing circuits shown in FIG.
  • 5C include only the n slave processing circuits in the first row, the n slave processing circuits in the m row, and the m slave processing circuits in the first column, that is, the k slave processing circuits are A plurality of slave processing circuits are directly connected to the master processing circuit.
  • K slave processing circuits are used for data and instruction forwarding between the master processing circuit and a plurality of slave processing circuits.
  • the main processing circuit may further include one or any combination of a conversion processing circuit 110, an activation processing circuit 111, and an addition processing circuit 112;
  • the conversion processing circuit 110 is configured to perform the interchange between the first data structure and the second data structure (such as conversion of continuous data and discrete data) on the data block or intermediate result received by the main processing circuit; or receive the main processing circuit
  • the data block or intermediate result performs the interchange between the first data type and the second data type (such as conversion of fixed-point types to floating-point types);
  • An activation processing circuit 111 configured to perform an activation operation on data in the main processing circuit
  • the addition processing circuit 112 is configured to perform an addition operation or an accumulation operation.
  • the main processing circuit is configured to determine that the input neuron is broadcast data, the weight value is distribution data, distribute the distribution data into a plurality of data blocks, and assign at least one of the plurality of data blocks and multiple data blocks. Sending at least one operation instruction among the two operation instructions to the slave processing circuit;
  • the plurality of slave processing circuits are configured to perform an operation on the received data block according to the operation instruction to obtain an intermediate result, and transmit the operation result to the master processing circuit;
  • the main processing circuit is configured to process a plurality of intermediate results sent from the processing circuit to obtain a result of the calculation instruction, and send the result of the calculation instruction to the controller unit.
  • the slave processing circuit includes: a multiplication processing circuit
  • the multiplication processing circuit is configured to perform a multiplication operation on a received data block to obtain a multiplication result
  • Forwarding processing circuit (optional) is used to forward the received data block or product result.
  • An accumulation processing circuit configured to perform an accumulation operation on the product result to obtain the intermediate result.
  • the operation instruction is a matrix multiplication by matrix instruction, an accumulation instruction, an activation instruction, and the like.
  • s s ( ⁇ wx i + b), where the weight w is multiplied by the input data x i , summed, and then offset After b, do the activation operation s (h) to get the final output s.
  • the computing unit includes: a tree module 40, the tree module includes: a root port 401 and a plurality of branch ports 404, the tree module The root port is connected to the main processing circuit, and the multiple branch ports of the tree module are respectively connected to one of the multiple slave processing circuits;
  • the above tree-type module has a transmitting and receiving function.
  • the tree-type module is a sending function.
  • the tree-type module is a receiving function.
  • the tree-type module is configured to forward data blocks, weights, and operation instructions between the master processing circuit and the multiple slave processing circuits.
  • the tree module is a selectable result of a computing device, which may include at least one layer node, the node is a line structure with a forwarding function, and the node itself may not have a computing function. If the tree module has zero-level nodes, the tree module is not needed.
  • the tree module may be an n-tree structure, for example, a binary tree structure as shown in FIG. 5F, or of course a tri-tree structure.
  • the n may be an integer greater than or equal to two.
  • the specific implementation of the present application does not limit the specific value of the foregoing n, and the above-mentioned number of layers may also be 2.
  • nodes of layers other than the penultimate node may be connected, for example, as shown in FIG. 5F Nodes in the penultimate layer.
  • the above-mentioned arithmetic unit may carry a separate buffer, as shown in FIG. 5G, which may include a neuron cache unit, the neuron cache unit 63 caches input neuron vector data and output neuron value data of the slave processing circuit .
  • the operation unit may further include: a weight buffer unit 64, configured to buffer weight data required by the slave processing circuit during calculation.
  • the arithmetic unit 12 may include a branch processing circuit 103 as shown in FIG. 5B, and a specific connection structure thereof is shown in FIG. 5B.
  • the main processing circuit 101 is connected to the branch processing circuit 103 (one or more), and the branch processing circuit 103 is connected to one or more slave processing circuits 102;
  • the branch processing circuit 103 is configured to execute data or instructions forwarded between the master processing circuit 101 and the slave processing circuit 102.
  • the controller unit obtains the input neuron matrix x, the weight matrix w, and the fully connected operation instruction from the storage unit, and transmits the input neuron matrix x, the weight matrix w, and the fully connected operation instruction to the main processing circuit;
  • the main processing circuit determines that the input neuron matrix x is broadcast data, determines the weight matrix w as distribution data, divides the weight matrix w into 8 sub-matrices, and then distributes the 8 sub-matrices to 8 slave processes through a tree module Circuit to broadcast the input neuron matrix x to 8 slave processing circuits,
  • the multiplication and accumulation operations of the 8 sub-matrixes and the input neuron matrix x are performed in parallel from the processing circuit to obtain 8 intermediate results, and the 8 intermediate results are sent to the main processing circuit;
  • the main processing circuit is configured to sort 8 intermediate results to obtain the operation result of wx, perform the operation of offset b on the operation result, and perform the activation operation to obtain the final result y, and send the final result y to the controller unit, the controller unit
  • the final result y is output or stored in a storage unit.
  • the method shown in FIG. 5A for a computing device to execute a forward operation instruction of a neural network may specifically be:
  • the controller unit extracts a neural network forward operation instruction, an operation domain corresponding to the neural network operation instruction, and at least one operation code from the instruction storage unit.
  • the controller unit transmits the operation domain to the data access unit and sends the at least one operation code. To the arithmetic unit.
  • the controller unit extracts the weight w and the offset b corresponding to the operation domain from the storage unit (when b is 0, it is not necessary to extract the offset b), and transmits the weight w and the offset b to the main processing unit. Circuit, the controller unit extracts the input data Xi from the storage unit, and sends the input data Xi to the main processing circuit.
  • the main processing circuit determines the multiplication operation according to the at least one operation code, determines that the input data Xi is broadcast data, determines that the weight data is distribution data, and divides the weight w into n data blocks;
  • the instruction processing unit of the controller unit determines the multiplication instruction, the offset instruction, and the accumulation instruction according to the at least one operation code, and sends the multiplication instruction, the offset instruction, and the accumulation instruction to the main processing circuit, and the main processing circuit inputs the multiplication instruction and the input data Xi is broadcasted to multiple slave processing circuits, and the n data blocks are distributed to the multiple slave processing circuits (for example, if there are n slave processing circuits, then each slave processing circuit sends a data block); multiple The slave processing circuit is configured to perform an multiplication operation on the input data Xi and the received data block according to the multiplication instruction to obtain an intermediate result, and send the intermediate result to the main processing circuit, which sends a plurality of slaves according to the accumulation instruction.
  • the intermediate result sent by the processing circuit performs an accumulation operation to obtain an accumulation result, and performs an addition offset b to obtain the final result according to the bias instruction, and sends the final result to the controller unit.
  • the technical solution provided in this application implements a multiplication operation and a bias operation of a neural network through a single instruction, that is, a neural network operation instruction.
  • the intermediate results calculated in the neural network do not need to be stored or retrieved, which reduces the storage and extraction operations of intermediate data. Therefore, it has the advantages of reducing the corresponding operation steps and improving the calculation effect of the neural network.
  • This application also discloses a machine learning computing device, which includes one or more computing devices mentioned in this application, which are used to obtain data to be calculated and control information from other processing devices, execute specified machine learning operations, and execute The result is passed to the peripheral device through the I / O interface.
  • Peripheral equipment such as camera, monitor, mouse, keyboard, network card, wifi interface, server.
  • the computing devices can link and transmit data through a specific structure, for example, interconnect and transmit data through the PCIE bus to support larger-scale machine learning operations.
  • a specific structure for example, interconnect and transmit data through the PCIE bus to support larger-scale machine learning operations.
  • its interconnection method can be any interconnection topology.
  • the machine learning computing device has high compatibility and can be connected to various types of servers through a PCIE interface.
  • FIG. 6A is a schematic diagram of a combined processing device.
  • Other processing devices include one or more types of processors such as a central processing unit CPU, a graphics processor GPU, and a neural network processor.
  • processors such as a central processing unit CPU, a graphics processor GPU, and a neural network processor.
  • the number of processors included in other processing devices is not limited.
  • Other processing devices serve as the interface between the machine learning computing device and external data and control, including data handling, to complete the basic control of the machine learning computing device such as start and stop; other processing devices can also cooperate with the machine learning computing device to complete the computing tasks.
  • a universal interconnection interface for transmitting data and control instructions between the machine learning computing device and other processing devices.
  • the machine learning computing device obtains required input data from other processing devices and writes it to the storage device on the chip of the machine learning computing device; it can obtain control instructions from other processing devices and write it to the control cache on the machine learning computing device chip;
  • the data in the storage module of the machine learning computing device can be read and transmitted to other processing devices.
  • the structure may further include a storage device, and the storage device is connected to the machine learning computing device and the other processing devices, respectively.
  • the storage device is configured to store data stored in the machine learning computing device and the other processing devices, and is particularly suitable for data that cannot be completely stored in the internal storage of the machine learning computing device or other processing devices.
  • the combined processing device can be used as an SOC system-on-chip for devices such as mobile phones, robots, drones, and video surveillance equipment, effectively reducing the core area of the control section, increasing processing speed, and reducing overall power consumption.
  • the universal interconnection interface of the combined processing device is connected to some parts of the equipment. Some components such as camera, monitor, mouse, keyboard, network card, wifi interface.
  • a chip is also applied, which includes the above-mentioned machine learning computing device or combined processing device.
  • a chip packaging structure is applied, which includes the above chip.
  • a board card is applied, which includes the chip package structure described above. Referring to FIG. 7B, FIG. 7B provides a board card. In addition to the above chip 389, the board card may also include other supporting components, which include, but are not limited to, a storage device 390, an interface device 391, and a control device 392 ;
  • the memory device 390 is connected to a chip in the chip package structure through a bus, and is used to store data.
  • the memory device may include a plurality of sets of memory cells 393. Each group of the storage units is connected to the chip through a bus. It can be understood that the memory cells in each group may be DDR SDRAM (English: Double Data Rate SDRAM, double-rate synchronous dynamic random access memory).
  • the storage device may include 4 groups of the storage units. Each group of the memory cells may include a plurality of DDR4 particles (chips). In one embodiment, the chip may include four 72-bit DDR4 controllers. Among the 72-bit DDR4 controllers, 64 bits are used for data transmission and 8 bits are used for ECC check. It can be understood that when DDR4-3200 particles are used in each group of the storage units, the theoretical bandwidth of data transmission can reach 25600MB / s.
  • each group of the storage units includes a plurality of double-rate synchronous dynamic random access memories arranged in parallel.
  • DDR can transfer data twice in one clock cycle.
  • a controller for controlling DDR is provided in the chip, and is used for controlling data transmission and data storage of each of the storage units.
  • the interface device is electrically connected to a chip in the chip package structure.
  • the interface device is used to implement data transmission between the chip and an external device (such as a server or a computer).
  • the interface device may be a standard PCIE interface.
  • the data to be processed is transferred from the server to the chip through a standard PCIE interface to implement data transfer.
  • the interface device may also be other interfaces.
  • the present application does not limit the specific expressions of the other interfaces described above, and the interface unit can implement the transfer function.
  • the calculation result of the chip is still transmitted by the interface device to an external device (such as a server).
  • the control device is electrically connected to the chip.
  • the control device is configured to monitor a state of the chip.
  • the chip and the control device may be electrically connected through an SPI interface.
  • the control device may include a microcontroller (Micro Controller Unit).
  • the chip may include multiple processing chips, multiple processing cores, or multiple processing circuits, which may drive multiple loads. Therefore, the chip can be in different working states such as multiple loads and light loads.
  • the control device can realize the regulation of the working states of multiple processing chips, multiple processes, and / or multiple processing circuits in the chip.
  • an electronic device which includes the board card described above.
  • Electronic equipment includes data processing devices, robots, computers, printers, scanners, tablets, smart terminals, mobile phones, driving recorders, navigators, sensors, cameras, servers, cloud servers, cameras, camcorders, projectors, watches, headphones , Mobile storage, wearables, transportation, home appliances, and / or medical devices.
  • the vehicles include airplanes, ships, and / or vehicles;
  • the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, cooker hoods;
  • the medical equipment includes nuclear magnetic resonance instruments, B-ultrasound and / or electrocardiograph.
  • the signal collector is used to receive signals, and transmit the signals and data to be processed to the chip inside the board as input data.
  • the image processing chip may be an artificial neural network processing chip.
  • the signal collector is a microphone or a multi-array microphone.
  • the chip inside the board includes the same embodiment as described above, which is used to obtain the corresponding output data (that is, the processed image) and transmit it to the interactive interface.
  • the interactive interface receives the output data of the chip (which can be regarded as an artificial neural network processor), and converts it into a suitable form of feedback information to display to the user.
  • the chip which can be regarded as an artificial neural network processor
  • the controller unit receives the user's operation or command and controls the operation of the entire signal processing device.
  • the electronic device may be a data processing device, a robot, a computer, a tablet computer, a smart terminal, a mobile phone, a cloud server, a camera, a video camera, a projector, a watch, a headset, a mobile storage, or a wearable device.
  • FIG. 9 is a schematic flowchart of a signal processing method according to an embodiment of the present invention. As shown in Figure 9, the method includes:
  • the signal processing device collects a signal input by a user.
  • the signal processing device converts the signal into a processing instruction and target data according to a target signal instruction conversion model, where the target data is processing data of data to be processed.
  • the converting the signal into a processing instruction and target data according to a target signal instruction conversion model includes:
  • the method is configured to perform area division on the data to be processed according to the granularity of the semantic region and the image recognition technology in the processing instruction to obtain the target data.
  • the converting the signal into a processing instruction and target data according to a target signal instruction conversion model includes:
  • the method further includes:
  • the text information or the processing instruction or the target data is stored.
  • the signal processing device processes the target data according to the processing instruction and a target processing model.
  • the processing the target data according to the processing instruction and a target processing model includes:
  • the processing the target data according to the M processing instructions and the target processing model includes:
  • the method before the receiving the signal and the data to be processed, the method further includes:
  • Adaptive training is performed on the signal instruction conversion model to obtain the target signal instruction conversion model.
  • the adaptive training of the signal instruction conversion model is performed offline or offline.
  • the adaptive training of the signal instruction conversion model is supervised or unsupervised.
  • the adaptively training the signal instruction conversion model to obtain the target signal instruction conversion model includes:
  • the signal instruction conversion model is optimized according to the correlation coefficient between the predicted instruction and its corresponding instruction set to obtain the target signal instruction conversion model.
  • the method before the collecting a signal input by a user, the method further includes:
  • Adaptive training is performed on the processing model to obtain the target processing model.
  • the adaptive training of the processing model is performed offline or offline.
  • the adaptive training of the processing model is supervised or unsupervised.
  • the adaptively training the processing model to obtain a target processing model includes:
  • the processing model is optimized according to a correlation coefficient between the prediction data and its corresponding training data to obtain the target processing model.
  • the method further includes:
  • An embodiment of the present invention also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program causes a computer to execute part or all of the steps of any method described in the foregoing method embodiments
  • the computer includes electronic equipment.
  • An embodiment of the present invention also provides a computer program product.
  • the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute any one of the methods described in the foregoing method embodiments. Part or all of the steps of a method.
  • the computer program product may be a software installation package, and the computer includes an electronic device.
  • the disclosed device may be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the unit is only a logical function division.
  • multiple units or components may be combined or may Integration into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be electrical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objective of the solution of this embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above integrated unit may be implemented in the form of hardware.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Data Mining & Analysis (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Molecular Biology (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Acoustics & Sound (AREA)
  • Medical Informatics (AREA)
  • Neurology (AREA)
  • Social Psychology (AREA)
  • Psychiatry (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

本发明公开了一种信号处理装置，包括：信号采集器、指令转换器和处理器。采用本发明实施例能够实现精准识别用户用意，给用户带来操作便捷性。

Description

信号处理装置、信号处理方法及相关产品 技术领域
本发明涉及数据处理领域，尤其涉及一种信号处理装置、信号处理方法及相关产品。
背景技术
随着电子设备(如手机、平板电脑、服务器等计算机设备)的大量普及应用，电子设备能够支持的应用越来越多，功能越来越强大，电子设备向着多样化、个性化的方向发展，成为用户生活中不可缺少的电子用品。
目前来看，语音识别、图像识别是电子设备的常用技术手段，但是，目前的电子设备尚不能精准识别用户的真实用意，因此，在实际应用过程中，给用户带来各种操作不便。
发明内容
本发明实施例提供一种信号处理装置、信号处理方法及相关产品，实现了输入信号即可对数据进行处理的功能，能够实现精准识别用户用意，给用户带来操作便捷性。
第一方面，本发明实施例提供一种信号处理装置，包括：
信号采集器，用于采集用户输入的信号；
指令转换器，用于根据目标信号指令转换模型将所述信号转换成处理指令和目标数据，所述目标数据为待处理数据的处理数据；
处理器，用于根据所述处理指令和目标处理模型对所述目标数据进行处理。
第二方面，本发明实施例还提供了一种机器学习运算装置，所述机器学习运算装置包括一个或多个如第一方面所述的信号处理装置，用于从其他处理装置中获取待运算输入数据和控制信息，并执行指定的机器学习运算，将执行结果通过I/O接口传递给其他处理装置；
当所述机器学习运算装置包含多个所述信号处理装置时，所述多个所述信号处理装置间可以通过特定的结构进行连接并传输数据；
其中，多个所述信号处理装置通过快速外部设备互连总线PCIE总线进行互联并传输数据，以支持更大规模的机器学习的运算；多个所述信号处理装置共享同一控制系统或拥有各自的控制系统；多个所述信号处理装置共享内存或者拥有各自的内存；多个所述信号处理装置的互联方式是任意互联拓扑。
第三方面，本发明实施例还提供了一种组合处理装置，所述组合处理装置包括如第二方面所述的机器学习运算装置，通用互联接口和其他处理装置；
所述机器学习运算装置与所述其他处理装置进行交互，共同完成用户指定的计算操作。
第四方面，本发明实施例还提供了一种神经网络芯片，所述机器学习芯片包括如第二方面所述的机器学习运算装置或如第三方面所述的组合处理装置。
第五方面，本发明实施例还提供了一种电子设备，所述电子设备包括如第四方面所述的芯片。
第六方面，本发明实施例还提供了一种板卡，其特征在于，所述板卡包括：存储器件、接口装置和控制器件以及如第六方面所述的神经网络芯片；
其中，所述神经网络芯片与所述存储器件、所述控制器件以及所述接口装置分别连接；
所述存储器件，用于存储数据；
所述接口装置，用于实现所述芯片与外部设备之间的数据传输；
所述控制器件，用于对所述芯片的状态进行监控。
第七方面，本发明实施例还提供了一种信号处理方法，包括：
采集用户输入的信号；
根据目标信号指令转换模型将所述信号转换成处理指令和目标数据，所述目标数据为待处理数据的 处理数据；
根据所述处理指令和目标处理模型对所述目标数据进行处理。
第八方面，本申请实施例提供了一种计算机可读存储介质，其中，上述计算机可读存储介质存储用于电子数据交换的计算机程序，其中，上述计算机程序使得计算机执行如本申请实施例第七方面中所描述的部分或全部步骤。
第九方面，本申请实施例提供了一种计算机程序产品，其中，上述计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质，上述计算机程序可操作来使计算机执行如本申请实施例第七方面中所描述的部分或全部步骤。该计算机程序产品可以为一个软件安装包。
可以看出，在本发明实施例的方案中，信号采集器采集用户输入的信号；指令转换器根据目标信号指令转换模型将信号转换成处理指令和目标数据，目标数据为待处理数据的处理数据；处理器根据所述处理指令和目标处理模型对所述目标数据进行处理。与现有的处理技术相比，本发明通过信号进行识别处理，能够实现精准识别用户用意，给用户带来操作便捷性。
本发明的这些方面或其他方面在以下实施例的描述中会更加简明易懂。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本发明实施例提供的一种信号处理装置的结构示意图；
图2为本发明实施例提供的另一种信号处理装置的局部结构示意图；
图3为本发明实施例提供的另一种信号处理装置的局部结构示意图；
图4A为本发明实施例提供的另一种信号处理装置的局部结构示意图；
图4B为本发明实施例提供的一种信号处理演示示意图；
图4C为本发明实施例提供的另一种信号处理演示示意图；
图5A是本申请实施例提供的一种计算装置的结构示意图；
图5B是本申请一个实施例提供的计算装置的结构图；
图5C是本申请另一个实施例提供的计算装置的结构图；
图5D是本申请实施例提供的主处理电路的结构图；
图5E是本申请实施例提供的另一种计算装置的结构图；
图5F是本申请实施例提供的树型模块的结构示意图；
图5G是本申请实施例提供的又一种计算装置的结构图；
图5H是本申请实施例提供的还一种计算装置的结构图；
图6A是本申请实施例提供的一种组合处理装置的结构图；
图6B是本申请实施例提供的一种计算装置的结构示意图；
图7A是本申请实施例提供的另一种组合处理装置的结构图；
图7B是本申请实施例提供的一种板卡的结构示意图；
图8为本发明实施例提供的一种电子设备的结构示意图；
图9为本发明实施例提供的一种信号处理方法的流程示意图。
具体实施方式
以下分别进行详细说明。
本发明的说明书和权利要求书及所述附图中的术语“第一”、“第二”、“第三”和“第四”等是用于区别不同对象，而不是用于描述特定顺序。此外，术语“包括”和“具有”以及它们任何变形，意图在 于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元，而是可选地还包括没有列出的步骤或单元，或可选地还包括对于这些过程、方法、产品或设备固有的其它步骤或单元。
在本文中提及“实施例”意味着，结合实施例描述的特定特征、结构或特性可以包含在本发明的至少一个实施例中。在说明书中的各个位置出现该短语并不一定均是指相同的实施例，也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是，本文所描述的实施例可以与其它实施例相结合。
请参见图1，图1为本发明实施例提供的一种信号处理装置的结构示意图。如图1所示，该信号处理装置100包括：
信号采集器1019，用于采集用户输入的信号。
其中，上述信号包括但不限于语音信号、图像信号、文本信号、感应信号等。该信号采集装置可以只采集其中一种信号，也可以支持采集多种信号。
可选地，上述信号处理装置100还包括信号过滤器，上述信号采集器101采集到上述信号后，上述信号过滤器对该输入的信号进行降噪处理，降低信号中无用数据的干扰。
可选地，该信号采集器可为语音信号采集器，可以包括但不仅限于：语音传感器、麦克风、拾音器，或者，获取其他音频采集装置，在此不作限定；该信号采集器还可为图像采集器，可以包括但不仅限于：照相机、摄像机及其他图像采集装置，在此不作限定；该信号采集器也可为感应信号采集器，可以包括但不仅限于：触摸屏、触控板、体感传感器、超声波传感器、脑电波传感器及其他感应信号采集装置。
具体地，以上述步骤101中的信号为语音信号为例，上述信号采集器1019可在接收语音信号时，还接收环境声音信号。上述信号过滤器可根据上述环境声音信号对上述语音信号进行降噪处理。该环境声音信号对上述语音信号来说是噪声。进一步地，上述该信号采集器101可包括对麦克风阵列，既可用于采集上述语音信号和上述环境声音信号，又实现了降噪处理。
具体地，以上述步骤101中的信号为图像信号为例，上述信号采集器1019可在接收图像信号时，图像信息中只有部分内容为有效信息，譬如手势、面部表情、动作趋势。上述信号过滤器用于过滤掉无用信息，仅仅提取出图像中的有用信息，如手势、面部表情、动作趋势。进一步地，上述该图像采集器101可包括体感传感器，可用于采集图像后提取出人的体感动作和指向进行定位，得到有用信息。
具体地，以上述步骤101中的信号为文本信号为例，上述信号采集器1019可在接收文本信号时，文本信息中并非全部内容均为有效信息。上述文本信号过滤器用于过滤掉无用信息，提取文本信号的有用信息，具体地，例如，提取关键字。
可选地，在一种可行的实施例中，上述信号处理装置还包括第一存储模块。上述信号采集器采集到上述信号后，上述信号处理装置将上述信号存储到第一存储模块中。
指令转换器1029，用于根据目标信号指令转换模型将所述信号转换成处理指令和目标数据，所述目标数据为待处理数据的处理数据。
其中，处理指令可以用于实现如下功能：数据修改、数据编辑、数据整合、数据拼接、数据删除等，在此不作限定。处理指令可以包括以下至少一种：图像处理指令、语音处理指令、文本处理指令等等，在此不作限定。以图像处理指令为例，可以用于实现如下功能：改变图像部分或全部颜色，改变图像部分或全部的亮度，改变图像的一部分或全部的对比度等，替换图像部分内容为其他内容，编辑图像部分或全部内容进行扩大/缩小，编辑部分内容或全部内容进行虚化或马赛克化，将多张图像进行拼接，将多张图像进行整合，删除图像中的部分内容等，在此不作限定，以语音处理指令为例，可以用于实现如下功能：改变语音数据的部分或全部响度、频率和音色，替换语音数据部分内容为其他内容，修改语音内容的部分或全部内容，删除语音数据部分内容并将剩余内容进行拼接，对多段语音数据进行整合等，在此不作限定。以文本处理指令为例，可以用于实现如下功能：删除文本文件、对文本文件进行编辑、创建文件夹、文件加密/解密、文件传输、文件压缩，等等，在此不作限定。本申请实施例中，数据可以为 以下至少一种：文本数据、语音数据、图像数据，在此不作限定。
本发明实施例中，处理指令可以定义如下三类处理指令，具体地：逻辑处理指令、精确处理指令、模糊处理指令，当然还可以包括其他处理指令。处理指令可至少包括操作类型、操作区域，当然，处理指令还可以包括操作数据。操作类型可以包括以下至少一种：图像识别、图像分割、图像增强、图像复原、图像去噪、背景虚化、图像拼接、图像整合、语音识别、语音合成、语音降噪、语音分离、文本截图、文本编辑等，在此不作限定。
逻辑处理指令，用于处理给定具有逻辑含义的操作需求，以针对图像的逻辑处理指令为例，如把图中的足球移走，该操作包括对图中足球的区域的检测和定位，将足球的区域删除，以及将该区域和区域周围的内容进行融合。该逻辑处理指令中可包括操作类型，逻辑操作区域，逻辑操作数据。
精确处理指令，用于处理给定精确数据的操作需求，如给定坐标范围，以针对图像的精确处理指令为例，如以图中心点为原点，半径为10个像素点的圆形区域内，改变该区间的颜色为RGB各值分别为255、0、0。该精准处理指令中可包括操作类型，精确操作区域，精确操作数据。
模糊处理指令，用于处理给出模糊数据的操作需求，如对“中心点附近”的区域进行操作。该模糊处理指令中可包括操作类型，模糊操作区域，模糊操作数据。
其他处理指令，包括对数据输入输出、数据判断等相关操作指令。
其中，上述操作类型可为编码的形式，通过某种编码方式定义而来，如依次按序编码的方式。操作类型包括但不限于移动，删除，替换，修改参数(如变色、亮度、对比度)，修改形状(缩小、放大、变形)等，具体而言，可包括改变操作区域的颜色，改变操作区域的亮度，改变操作区域的对比度等，删除操作区域的内容，替换操作区域的内容为其他内容，扩大/缩小/旋转操作区域的内容，虚化/雾化/马赛克化操作区域的内容，合成操作区域，拼接操作区域，对操作区域建模等，在此不作限定。
其中，上述操作区域可为整幅图像，可以为图像中的部分区域，根据其给定的方式可包括上述逻辑区域、精确区域、模糊区域或者其他区域。逻辑区域可指为具有某种特定的逻辑含义的区域，包括指定物品(如足球、书桌、小狗等)、指定背景(如海、山等)、指定部分(如人脸、骨骼框架、鸟的翅膀等)等。这些逻辑区域采用机器学习等算法进行离线训练而得，并且可以通过在线训练的方式，为用户定制个性化方案。精确区域可指具有精确定位的区域，包括精确坐标及范围，精确要求及范围。譬如，以图中心点为原点，半径为10个像素点的圆形区域；图中所有颜色RGB值为(255，0，0)的位置等。模糊区域为给出一个指向但定位和范围略有模糊的区域，如图像左上角，红色系的地方等。该部分内容的定位和确定可以通过机器学习算法离线训练得到的选取模型，针对待处理图像的特征，智能选取待处理的范围。
可选地，操作指令中还包括操作数据，包括逻辑操作数据、精确操作数据、模糊操作数据。逻辑操作数据具有某种特定的逻辑含义的数据，包括指定物品(如足球、书桌、小狗等)、指定背景(如海、山等)、指定部分(如人脸、骨骼框架、鸟的翅膀等)等。精确数据可包括精确的数值参数或者指向数据的精确地址，以逻辑指令类为例，逻辑指令：人脸换足球，操作类型：替换，操作区域：人脸区域，操作数据：足球(或指向最求图形的地址)。模糊数据可包括模糊的描述，如颜色“深一点”、亮度“暗一点”等。
举例说明下，输入单张图像，输出单张图像。信号采集器，接收语音信号，并进行预处理，降低无效信息，如噪音等，将经过预处理后的语音信号送入指令转换单元，利用语音识别技术，对语音信号进行解析，并将其转换为预先定义的图像处理指令，图像处理单元接收图像处理指令，根据指令利用图像识别技术对待处理图像进行定位和划分，得到待处理图像的目标区域，如果需要操作数据，则从存储单元中读取所需要的操作数据，送入图像处理单元。图像处理单元根据处理指令，利用操作数据对待处理图像的目标区域完成给定操作，操作完成，输出图像。
举例说明下，给出该处理的一个实例，该实例为输入的待处理数据为一幅站立的人像图片，输入的处理要求是一条语音信号，即“换背景，站到海边”，接收一幅人像图像作为待处理数据，而后送入待 处理器等待处理，信号处理器1019接收通过麦克风阵列接收语音信号并进行降噪处理后得到有效语音要求即“换背景，站到海边”，指令转换器1029对传入的语音信号利用语音识别技术将语音信号转化成为了文本信息，利用自然语言处理技术将文本信息转化成为了处理指令和目标区域(目标数据)，说明待处理的操作为替换，目标区域为待处理图像中的“人”和图像库中的海边图中的“背景”，而后将解析后的内容传入存储器1049中暂时存储，处理器1039从存储模块中提取待处理数据和处理指令解析出的目标区域，而后利用图像处理技术对待处理图像进行区域划分，得到相对应的目标区域，存储到存储器1049中；从存储器1049中搜索选出所需的已知数据，即海边背景图，利用图像处理技术对已知数据进行区域划分，得到相对应的已知数据的目标区域，存储到存储器1049中；从存储器1049中获取待处理数据的目标区域和已知数据的目标区域，对二者进行智能整合，即对人像根据背景进行适当的缩放，根据预设的环境，即需要一个庄重而严肃的姿势和表情，适当变换人像的姿势和表情，而后将其整合到背景图中，并进行边缘柔化、根据阳光加人物阴影等操作得到处理后的图像，如果接收到用户发出的结束的信息，则处理过程结束，输出数据即可；否则继续接受用户的处理请求，直到接收到用户的结束的信息。其中，不限制结束信息的形式，可以为语音形式、图像形式，也可以通过令用户通过点击结束按键得到信息的形式。譬如，用户对姿势不满意，想要换为另一幅图像中人物的姿势，此时用户可以在输入一幅新的图像作为参考图像，并输入语音要求，信号采集器1019采集语音要求并送入处理器1039对要求进行解析，根据解析结果，获取参考图像的目标区域，及参考图像的人像姿势。检验用户需求是否合理，即参考图像的人像姿势非常不适合摆在海边前，那么处理器1039主动向用户发送一条确认信息，询问用户是否确定进行该操作。如检测不存在不合理的情况或者用户确定进行该操作，那么根据参考人像姿势编辑原处理后的图像中的人像姿势，而后进行边界柔化等处理获得新的处理后的图像，在此过程中，用户的需求可以采用多种输入形式相结合的方式，举例来说，当用户此时希望移动人物位置，通过语音要求“人”对人物整体进行选定作为目标区域，而后通过摄像头获取用户手势，对人物进行左右移动。类似地，直到接收到用户发出的结束的信息，则处理过程结束。
可选地，上述指令转换器1029在根据语音识别技术、自然语言处理技术和图像识别技术等技术将所述信号转换成处理指令和目标数据之前，上述指令转换器1029从上述第一存储模块中获取上述步骤101中的信号。
其中，如图2所示，所述指令转换器1029包括：
第一信号识别器1021，用于将所述信号转换成文本信息。可选的，该识别器可包含一个或多个子识别器，如语音信号子识别器通过语音识别技术将语音信号转换成文本信息，图像信号子识别器通过图像识别技术将一幅图像或一组图像的有效信号转换成文本信息等；
可选的，在将所述信号转换成文本信息的过程中还可以同时处理多种类型信号，如同时输入语音信号和图像信号，分别利用语音信号子识别器和图像信号子识别器对其进行识别。
图像识别技术，即对输入的图像信号进行识别和分析，可包括但不限于以下算法：利用OTSU算法、Roberts算子、Sobel算子、Normalized Cuts算法进行图像分割、利用Bag of words(BoW)方法、Partsand structure方法、生成性(generative)方法与鉴别性(Discriminative)方法等方法进行目标识别和目标检测，利用核相关滤波(KDF)算法、DSST(Discriminative Scale Space Tracker)算法等进行目标跟踪。用以从一张或一组图像中提取出图像信号所表达的操作。
举例说明下，上述图像信号处理一组图像的过程。具体地，假设利用摄像头采集多幅图像，利用图像信号子识别器，对采集到的图像进行分割，剔除无效信息，即将图像的前景和背景进行分割，剔除背景信息，获取前景信息，接着，确定一个或多个目标的位置及大小等相关信息，具体地，如进行利用边缘检测和分割技术，如：边缘检测算法：sobel边缘检测算法、laplace边缘检测算法等，又如分割算法：基于阈值的图像分割算法、基于边缘的图像分割算法、基于区域的边缘分割算法、基于图论的边缘分割算法、基于能量泛函的分割算法、基于小波变换的分割算法、基于神经网络的分割算法，从而，确定人脸、人手、人的骨骼架构的位置及范围；其次，对目标进行识别并提取特征，特征提取技术可包括SIFT、 SURF等角点检测包括HARRIS fast等，如果具有多个备选目标，则确定各备选目标备选的优先级；最后，筛选出有效目标，并利用目标追踪技术，如：相关滤波和深度学习方法，在此不做限定，追踪目标运动轨迹，得到该目标所表达的含义，如获取到人脸、人的骨骼架构基本没有变化，而人手从左边滑到右边，得到该有效信息是将待处理图像的目标区域从左边移动到右边，而后将有效的图像信息将其转化为文本信息。
再举例说明下，上述文本信号处理一段文本的过程。假设利用文本信息采集器获取到了一段文本信息，利用文本信号子识别器，对采集到的文本进行切分，形成文本矢量。而后利用文本特征提取技术，文本特征提取技术可以包括以下至少一种：TF-IDF、词频方法、文档频次方法、互信息、期望交叉熵、二次信息熵、二次信息熵、x2统计量方法、文本证据权、遗传算法、主成分分析法、模拟退火算法、N—Gram算法等等，在此不做限定，筛选出有效文本信息。结合离线训练而成的文本匹配模型，将有效的采集文本信息转化成为可送入信号文本转换器的文本信息。
再举例说明下，上述多种类型信号的处理过程。假设两个或以上信号采集器采集到信息，假设是语音信息和图像信息，那么将语音信息通过语音信号子识别器处理成为了语音信号对应的文本信息，将图像信息通过图像信号子识别器处理成为了图像信号对应的预设类型信息，预设类型信息可以为以下至少一种：文本信息、图像信息、语音信息、投影信息、振动信息等等，在此不作限定。
信号文本转换器1022，用于将所述一条或多条文本信息通过自然语言处理技术和所述目标信号指令转换模型转换成所述处理指令；
第一数据识别器1023，用于根据所述处理指令中的语义区域的粒度和图像识别技术对所述待处理数据进行区域划分，得到所述目标数据。
进一步地，上述指令转换器1029还包括：
获取模块1026，用于获取上述处理指令中的语义区域的粒度。
举例说明上述语义区域，假设上述信号处理装置100根据语音信号确定对上述目标数据为人脸区域时，则上述语义区域为上述待处理数据中的人脸区域，上述信号处理装置以人脸为粒度，获取上述待处理数据中的多个人脸区域；当上述目标数据为背景，上述信号处理装置将上述待处理数据划分为背景区域和非背景区域；当上述目标数据为红颜色区域时，上述信号处理装置将上述待处理数据按照颜色划分为不同颜色的区域。
可选的，上述指令转换器1029还包括交互模块，用于完成系统与用户之间的交互操作，可以通过发送交互信息给用户，接收用户反馈信息，并依据该反馈信息生成相应的指令，可将该指令传输给处理器1039执行相应操作，交互信息可以为提示用户进行指定操作的指示信息，指定操作可以由用户自行设置或者系统默认，反馈信息可以由用户输入，输入方式可包括但不仅限于：语音输入、触控输入、肢体动作输入等等，在此不做限定。例如，交互模块用户判断目标区域和处理指令是否符合判断模型的预设要求，预设要求可以由用户自行设置，或者，系统默认。例如，要求头脚互换，则不符合常理。如果不符合，返回提示信息给用户，如询问用户是否确定执行该操作。其中，判断模型是通过离线训练得到的模型，也可以是用户自定义得到的模型。如收到确认的信息，则继续执行。用户确认的方式可以包括以下至少一种：可通过点击按钮、语音输入、图像输入等多种方式，在此不作限定。又如交互模块用于根据预设顺序或根据用户习惯，依次缩放待处理图像的一个局部区域和/或几个局部区域，便于进行更为细节的操作，具体地，如用户习惯为修改人像图像时，依次眉、眼、嘴的调整顺序，那么该装置检测到待处理图像为人像图像时，依次取眉、眼、嘴部分进行放大，便于用户对其进行操作。
具体地，本发明中使用的语音识别技术包括但不限于采用人工神经网络(Artificial Neural Network，ANN)、隐马尔科夫模型(Hidden Markov Model，HMM)等模型，上述第一信号识别器中的语音信号子识别器可根据上述语音识别技术处理上述语音信号；上述自然语言处理技术包括但不限于利用统计机器学习、ANN等方法，上述语义理解单元可根据上述自然语言处理技术提取出语义信息；上述图像识别技术包括但不限于利用基于边缘检测的方法、阈值分割方法、区域生长与分水岭算法、灰度积分投影曲线 分析、模板匹配、可变形模板、Hough变换、Snake算子、基于Gabor小波变换的弹性图匹配技术、主动形状模型和主动外观模型等方法等算法，上述图像信号子识别器可根据上述图像识别技术将上述待处理数据分割成不同的区域。
上述自然语言处理技术，即让计算机能够理解输入的语音或者文本信息。包括但不限于最大匹配算法，word2vec算法，doc2vec算法，潜在语音索引算法，潜在狄利克雷分布算法，以及机器学习算法，包括支持向量机、神经网络、AdaBoost等算法。用以完成词法分析、语法分析、语义分析、情感倾向性分析等，得到输入信号中所表达的观点和意义。譬如，读取信号数据之后，对数据进行去噪音。而后对数据进行分词和标注，利用TF-IDF、word2vec等算法建立词袋模型。利用WLLR(Weighted Log Likelihood Ration)加权对数似然、WFO(Weighted Frequency and Odds)加权频率和可能性等算法对特征进行选择。得到输入信号的特征。根据该特征直接转换为相应的文本信息，或者根据该特征和预训练而成的模型得到相应的文本信息。譬如，以语义理解技术为例，利用语义树、主题模型、词向量等技术对输入信号的语义进行分析，得到输入信号所表达的主题和操作，便于后续选择合适的模型进行处理。
举例说明下，给出该处理的另一个实例，该实例为输入的待处理数据为一段语音，参考数据是一段文本，输入的处理要求是一段语音，要求将文本读出来插在待处理语音中间。信号采集器1019接收一段语音，经过降噪等预处理后作为待处理数据，送入存储器1049中暂时存储。信号采集器1019接收通过麦克风阵列接收语音信号并进行降噪处理后得到有效语音要求，送入处理器1039。处理器1039对传入的语音信号利用语音识别技术直接将语音信号转化成为了待处理数据处理指令和目标区域，根据待处理数据处理指令，利用文本转语音技术将输入的文本形式的参考数据转换成为参考语音，而后从存储器1049中读取待处理数据，根据待处理数据的语音的语音特性，如语音语调，对参考语音进行处理。最后，将参考语音插入到待处理语音中的目标区域内，并进行平滑处理，得到处理后的语音数据。如果接收到用户发出的结束的信息，则处理过程结束，数据即可，否则继续接受用户的处理请求，直到接收到用户的结束的信息。其中，不限制结束信息的形式，不限制结束信息的形式，可以为语音形式、图像形式，也可以通过令用户通过点击结束按键得到信息的形式。
再举例说明下，给出该处理的又一个实例，该实例为输入的待处理数据为一段中文文本，输入的处理要求是一段语音，要求将中文文本翻译成为一段适用于正式场合的英语语音。指令转换模块1029将语音处理要求解析成为了操作指令和目标区域。处理器1039根据输入的语音要求，首先将中文文本翻译成为正式用词和正式语法的英文文本，而后通过文本转语音的相关技术，调用存储器1049中处理后的语音数据库，将英文文本转换为较为正式的语音语调的语音数据。如果接收到用户发出的结束的信息，则处理过程结束，数据即可；否则继续接受用户的处理请求，直到接收到用户的结束的信息。其中，不限制结束信息的形式，不限制结束信息的形式，可以为语音形式、图像形式，也可以通过令用户通过点击结束按键得到信息的形式。
在一种可行的实施例中，上述第一信号识别器1021将上述信号通过上述语音识别技术、图像识别技术、自然语言处理等转换成有效的文本信息，并将该文本信息保存到上述第一存储模块中。上述信号文本转换器1022从上述第一存储模块中获取上述文本信息，并将该文本信息通过自然语言处理技术和上述目标信号指令转换模型转换成处理指令，并将上述处理指令保存到上述第一存储模块中；上述第一数据识别器1023根据上述处理指令中的语义区域的粒度和图像识别技术对上述待处理数据进行区域划分，得到上述目标数据，并将上述划分结果和上述目标数据存储到上述第二存储模块中。
在一种可行的实施例中，如图3所示，上述指令转换器1029也可以包括：
第二信号识别器1025，用于根据语音识别技术、自然语言处理技术和上述目标信号指令转换模型将上述信号直接转化为上述处理指令，并将该处理指令保存到第一存储模块中；
可选地，第二信号识别器1025包括一个或多个子识别器，如语音信号子识别器通过语音识别技术来识别语音信号，图像信号子识别器通过图像识别技术来识别一幅图像或多幅图像的有效信号。
举例说明下，上述图像信号处理一组图像的过程。具体地，假设利用摄像头采集多幅图像，利用图 像信号子识别器，对采集到的图像进行分割，剔除无效信息，即将图像的前景和背景进行分割，剔除背景信息，获取前景信息，接着，确定一个或多个目标的位置及大小等相关信息，具体地，如进行利用边缘检测技术，确定人脸、人手、人的骨骼架构的位置及范围；其次，对目标进行识别，如果具有多个备选目标，则确定各备选目标备选的优先级；最后，筛选出有效目标，并利用目标追踪技术，追踪目标运动轨迹，得到该目标所表达的含义，如获取到人脸、人的骨骼架构基本没有变化，而人手从左边滑到右边，得到该有效信息是将待处理数据的目标数据从左边移动到右边，而后将有效的图像信息将其转化为文本信息。
再举例说明下，上述文本信号处理一段文本的过程。假设利用文本信息采集器获取到了一段文本信息，利用文本信号子识别器，对采集到的文本进行切分，形成文本矢量。而后利用文本特征提取技术，筛选出有效文本信息。结合离线训练而成的文本匹配模型，将有效的采集文本信息转化成为可送入信号文本转换器的文本信息。
再举例说明下，上述多种类型信号的处理过程。假设两个或以上信号采集器采集到信息，假设是语音信息和图像信息，那么将语音信息通过语音信号子识别器处理成为了语音信号对应的文本信息，将图像信息通过图像信号子识别器处理成为了图像信号对应的文本信息。
第二数据识别器1025，根据上述处理指令对上述待处理数据进行操作的语义区域的粒度，将该待处理数据按照该语义区域的粒度进行划分，获取目标数据，该目标数据为对上述待处理数据进行处理的区域，并将划分后的结果以及上述目标数据存储到第二存储模块中。
可选地，在上述信号采集器1019接收上述信号和上述待处理数据之前，上述指令转换器1029对信号指令转换模型进行自适应训练，以得到上述目标信号指令转换模型。
其中，上述对信号指令转换模型进行自适应训练是离线进行的或者是在线进行的。
具体地，上述对信号指令转换模型进行自适应训练是离线进行的具体是上述指令转换器1029在其硬件的基础上对上述信号指令转换模型进行自适应训练，以得到目标信号指令转换模型；上述对信号指令转换模型进行自适应训练是在线进行的具体是一个不同于上述信号转换器102的云端服务器对上述信号指令转换模型进行自适应训练，以得到目标信号指令转换模型。上述指令转换器1029在需要使用上述目标信号指令转换模型时，该指令转换器1029从上述云端服务器中获取该目标信号指令转换模型。
可选地，上述对信号指令转换模型进行自适应训练是有监督的或者是监督的。
具体地，上述对上述信号指令转换模型进行自适应训练是有监督的具体为：
上述指令转换器1029根据信号指令转换模型将上述信号换成预测指令；然后确定上述预测指令与其对应的指令集合的相关系数，该指令集合为人工根据信号得到的指令的集合；上述指令转换器1029根据所述预测指令与其对应的指令集合的相关系数优化所述信号指令转换模型，以得到所述目标信号指令转换模型。
在一种可行的实施例中，上述信号处理装置100还包括：
训练器1059，用于根据所述信号指令转换模型将所述信号换成预测指令；确定所述预测指令与其对应的指令集合的相关系数；根据所述预测指令与其对应的指令集合的相关系数优化所述信号指令转换模型，以得到所述目标信号指令转换模型。
举例说明，上述对信号指令转换模型进行自适应训练是有监督的具体包括：上述指令转换器1029或者训练器1059接收一段包含相关命令的语音信号，如改变图像的颜色、旋转图片等。每种命令对应一个指令集合。对用于自适应训练的输入的语音信号来说，对应的指令集合是已知的，上述指令转换器1029或者训练器1059以这些语音信号作为信号指令转换模型的输入数据，获取输出后的预测指令。上述指令转换器1029或者训练器1059计算上述预测指令与其对应的指令集合的相关系数，并根据该自适应地更新上述信号指令转换模型中的参数(如权值、偏置等等)，以提高上述信号指令转换模型的性能，进而得到上述目标信号指令转换模型。
又以图像信号为例，上述指令转换器1029或者训练器1059接收一段包含相关命令的图像信号，如 通过手势指定图像中某物体，而后要求将其向右移动等。每种命令对应一个指令集合。对用于自适应训练的输入的图像信号来说，对应的指令集合是已知的，上述指令转换器1029或者训练器1059以这些图像信号作为信号指令转换模型的输入数据，获取输出后的预测指令。上述指令转换器1029或者训练器1059计算上述预测指令与其对应的指令集合的相关系数，并根据该自适应地更新上述信号指令转换模型中的参数(如权值、偏置等等)，以提高上述信号指令转换模型的性能，进而得到上述目标信号指令转换模型。
所述信号处理装置100还包括：
存储器1049，用于存储所述文本信息或者所述处理指令或者所述目标数据。
在一种可行的实施例中，上述存储器1049与上述第一存储模块和第二存储模块可以是同一个存储模块，还可以是不同的存储模块。
处理器1039，用于根据所述处理指令和目标处理模型对所述待处理数据进行处理。
在一种可行的实施例中，在所述根据目标信号指令转换模型将所述信号转换成处理指令和目标数据，所述目标数据为待处理数据的处理数据方面，所述指令转换器1029具体用于：
根据目标信号指令转换模型将所述信号转换成图像处理指令和目标区域，所述目标区域为待处理图像的处理区域；
在所述根据所述处理指令和目标处理模型对所述目标数据进行处理方面，所述处理器1039具体用于：
根据所述图像处理指令和目标图像处理模型对所述目标区域进行处理。
在一种可行的实施例中，在所述根据目标信号指令转换模型将所述信号转换成处理指令和目标数据，所述目标数据为待处理数据的处理数据方面，所述指令转换器1029具体用于：
根据目标信号指令转换模型将所述信号转换成图像处理指令和目标区域，所述目标区域为待处理图像的处理区域；
在所述根据所述处理指令和目标处理模型对所述目标数据进行处理方面，所述处理器1039具体用于：
处理器1039，用于根据所述图像处理指令和目标图像处理模型对所述待处理图像进行处理，得到处理图像，将所述处理图像转化为输出信息。
其中，本申请实施例在针对图像处理时，处理指令可以为图像处理指令，目标处理模型可以为目标图像处理模型。
可选地，在所述将所述处理图像转化为输出信息方面，所述处理器1039具体用于：
获取转换指令，所述转换指令包含指定数据类型，所述指定数据类型包括以下一种类型：语音类型、图像类型、文本类型；
在所述指定数据类型为图像类型时，输出所述处理图像；
在所述指定数据类型为文本类型时，输出文本信息；
在所述指定数据类型为语音类型时，输出语音信息。
其中，上述指定数据类型可以包括但不仅限于：包括以下一种类型：语音类型、图像类型、文本类型，在此不作限定，指定数据类型可以由用户自行设置或者系统默认。在指定数据类型为图像类型时，直接输出处理图像，在指定数据类型为文本类型时，输出文本信息，在指定数据类型为语音类型时，则输出语音信息。
其中，如图4A所示，所述处理器1039包括：
取指令模块1031，用于在预设时间窗口内从所述存储模块中获取M条处理指令，所述M为大于1的整数；
处理模块1032，用于根据所述M条处理指令和所述目标处理模型对所述目标数据进行处理。
可选地，所述处理模块1032用于：
删除所述M条处理指令中，功能相同的处理指令，以得到N条处理指令，所述N为小于所述M的整数；
根据所述N条处理指令和所述目标处理模型对所述目标数据进行处理。
具体地，上述预设时间窗口可以理解成预设时长。在预设时长内上述获取单元1031从上述存储模块104中获取M条处理指令后，上述处理模块1032对上述M条处理指令进行两两比较，将该M条处理指令中功能相同的指令删除，得到N条处理指令。上述处理模块1032根据上述N条处理指令和上述目标处理模型对上述待处理数据进行处理。
举例说明，上述处理模块1032对上述M条处理指令进行两两比较。当处理指令A和处理指令B一样时，上述处理模块1032删除上述处理指令A和B中开销最大的一条；当处理指令A和处理指令B不一样时，上述处理模块1032获取上述处理指令A和上述处理指令B的相似系数。当该相似系数大于相似阈值时，确定上述处理指令A和上述处理指令B功能相同，上述处理模块1032删除上述处理指令A和B中开销最大的一条；当上述相似系数小于上述相似阈值时，上述处理模块1032确定上述处理指令A和B的功能不同。该处理指令A和B为上述M条处理指令中的任意两条。
具体地，针对上述处理器1039，其输入和输出均为图像。上述处理器1039可以通过包括但不限定于ANN和传统计算机视觉方法对上述待处理数据进行的处理包括但不局限于：美体(例如美腿，隆胸)，换脸、美化脸，换物体(猫换狗、斑马变马，苹果换桔子等)，换背景(后面的森林换成田野)，去遮挡(例如人脸遮住了一个眼睛，重新把眼睛重构出来)，风格转换(一秒钟变梵高画风)，位姿转换(例如站着变坐着，正脸变侧脸)、非油画变油画、更换图像背景的颜色和更换图像中物体所处的季节背景、图像合成、图像融合、消除物体或阴影、图片着色、图片压缩、图片矢量化等。
再举例说明下，如图4B所示，对于输入多张图像的情况，图像依次以每次一张或多张进行处理，例如6张图像进行拼接，每次拼接2张。分别用图像1～6进行标记，
A1、信号采集单元接收语音信号，并进行预处理，降低无效信息，如噪音等；
A2、将经过预处理后的语音信号送入指令转换单元，利用语音识别技术，对语音信号进行解析，并将其转换为预先定义的图像处理指令；
A3、图像处理单元接收图像处理指令，根据指令利用图像识别技术对一张或多张待处理图像进行定位和划分，得到各自对应的待处理的目标区域；
A4、如果需要操作数据，则从存储单元中读取所需要的操作数据，送入处理单元；
A5、图像处理单元根据处理指令，利用操作数据对待处理图像的目标区域完成给定操作；
A6、如果图像未完全处理完毕，将处理后的图像保存到存储单元中进行存储。
进一步地，若读入新的一张或多张图像，则可以重复步骤A3～A6。直到全部图像处理完毕，读入各次的处理后的结果，根据图像处理指令，对各次的处理后的结果进行处理，得到最终的处理结果，输出图像。
再举例说明下，如图4C所示，以图像合成为例，这里采用曝光合成的方式，有6幅不同曝光程度的图像，每次处理2幅图像。
B1、信号采集单元，接收语音信号，并进行预处理，降低无效信息，如噪音等；
B2、将经过预处理后的语音信号送入指令转换单元，利用语音识别技术，对语音信号进行解析，并将其转换为预先定义的图像处理指令；
B3、图像处理单元接收图像处理指令，根据指令利用图像识别技术对一张或多张待处理图像进行定位和划分，得到各自对应的待处理的目标区域；
B4、如果需要操作数据，则从存储单元中读取所需要的操作数据，送入图像处理单元；
B5、图像处理单元根据处理指令，利用操作数据对待处理图像的目标区域完成给定操作；
B6、如果图像未完全处理完毕，将处理后的图像保存到存储单元中进行存储。
进一步地，若读入新的一张或多张图像，并将上一步的处理后的图像作为新图像一同处理，重复B3～B5，当全部图像处理完毕，B5得到的即为最终的处理结果，输出图像。
可选地，对于输入一张或多张图像，可根据图像处理模型对图像进行处理，得到所需要的文本信息，并输出，从而，实现如下功能，例如：图像分类、图像识别、图像分割、图像描述等功能，在此不作限定。其中，图像处理模型可用于图像分类、图像识别、图像分割或者图像描述等一种或多种功能。
举例说明下，输入图像，输出分类信息，具体如下：
C1、信号采集单元接收语音信号，并进行预处理，降低无效信息，如噪音等；
C2、将经过预处理后的语音信号送入指令转换单元，利用语音识别技术，对语音信号进行解析，并将其转换为预先定义的图像处理指令；
C3、图像处理单元接收图像处理指令，从存储单元读取预先存入的图像处理模型，利用图像处理模型对输入图像进行处理，得到分类结果；
C4、操作完成，将分类结果输出即可。
再举例来说，输入图像，输出图像描述，具体如下：
D1、信号采集单元接收语音信号，并进行预处理，降低无效信息，如噪音等。
D2、将经过预处理后的语音信号送入指令转换单元，利用语音识别技术，对语音信号进行解析，并将其转换为预先定义的图像处理指令；
D3、图像处理单元接收图像处理指令，从存储单元读取预先存入的图像处理模型对输入图像进行划分，提取出一个或多个目标区域，通过对目标区域进行处理，得到完整的文本描述。
D4利用需要输出语音描述，则利用语音转换技术，将文本描述转换为语音，输出即可；否则可以直接输出。操作完成。
再举例说明下，当然，输入的也可以不是语音信息，但输出的是图像信息，如输入文本/语音消息，或者，输入文本/语音消息和图像消息，将文本/语音消息转化为新图像，或者将文本/语音消息的内容对原图像消息进行添加或修改。
输入语音消息，根据图像处理指令，利用语音识别技术将语音消息转换成文本消息(若输入文本消息，则可以直接进行后续处理)，将文本消息按照指定或默认的图像规格、字体字号、背景颜色和花纹等要求进行处理，得到一幅新图像，输出即可，例如，将文本内容转换成图像的功能。可选地，输入的文本消息或者从语音消息转换而成的文本消息可先进行预处理，去除噪音、明显的错误等，可以提升后续处理精度。
输入语音消息，根据图像处理指令，利用语音识别技术将语音消息转换成文本消息(若输入文本消息，则可以直接进行后续处理)，将文本消息按照指定或默认的字体字号、背景颜色和花纹等要求进行处理，并添加到原图像的指定为位置，输出即可，例如弹幕功能。可选地，输入的文本消息和刚从语音消息转换而成的文本消息可先进行预处理，去除噪音、明显的错误等，可以提升后续处理精度。
输入文本信息，根据图像处理指令，对文本信息进行拆分。通过自然语音处理技术，分别对拆分后的文本消息进行处理并将其中的内容转化为对应的图像，例如，原本输入了图像，则在原图像中进行添加或修改，否则生成一幅新图像，而后根据转化的生成的图像进行拼接整合，得到与整段文字相对应的图像内容，输出即可。如果输入的是语音消息，可以将语音消息通过语音识别技术转化成为文本消息，也可以直接对语音消息进行理解分析，得到相对应的图像信息。
可选地，在上述信号采集器1019接收上述信号之前，上述处理器1039对处理模型进行自适应训练，以得到上述目标处理模型。
其中，上述对处理模型进行自适应训练是离线进行的或者是在线进行的。
具体地，上述对处理模型进行自适应训练是离线进行的具体是上述处理器1039在其硬件的基础上对上述处理模型进行自适应训练，以得到目标信号指令转换模型；上述对处理模型进行自适应训练是在线进行的具体是一个不同于上述处理器1039的云端服务器对上述处理模型进行自适应训练，以得到目 标处理模型。上述处理器1039在需要使用上述目标处理模型时，该处理器1039从上述云端服务器中获取该目标处理模型。
可选地，上述对处理模型进行自适应训练是有监督的或者是监督的。
具体地，上述对上述处理模型进行自适应训练是有监督的具体为：
以语音信号为例，上述处理器1039根据处理模型将上述语音信号换成预测数据；然后确定上述预测数据与其对应的训练数据的相关系数，该训练数据为人工根据语音信号对待处理数据进行处理得到的数据；上述处理器1039根据所述预测数据与其对应的训练数据的相关系数优化所述处理模型，以得到所述目标处理模型。
以图像信号为例，上述处理器1039根据处理模型将上述图像信号换成预测数据；然后确定上述预测数据与其对应的训练数据的相关系数，该训练数据为人工根据语音信号对待处理数据进行处理得到的数据；上述处理器1039根据所述预测数据与其对应的训练数据的相关系数优化所述处理模型，以得到所述目标处理模型。
在一种可行的实施例中，上述信号处理装置100还包括：
训练器1059，用于根据所述信号指令转换模型将所述信号换成预测指令；确定所述预测指令与其对应的指令集合的相关系数；根据所述预测指令与其对应的指令集合的相关系数优化所述信号指令转换模型，以得到所述目标信号指令转换模型。
举例说明，上述对处理模型进行自适应训练是有监督的具体包括：以语音信号为例，上述处理器1039或者训练器1059接收一段包含相关命令的语音信号，如改变图像的颜色、旋转图片等。每种命令对应一张训练数据。对用于自适应训练的输入的语音信号来说，对应的训练数据是已知的，上述处理器1039或者训练器1059以这些语音信号作为处理模型的输入数据，获取输出后的预测数据。上述处理器1039或者训练器1059计算上述预测数据与其对应的训练数据的相关系数，并根据该自适应地更新上述处理模型中的参数(如权值、偏置等等)，以提高上述处理模型的性能，进而得到上述目标处理模型。
以图像信号为例，上述处理器1039或者训练器1059接收一段包含相关命令的图像信号，如改变图像的颜色、旋转图片等。每种命令对应一张训练数据。对用于自适应训练的输入的语音信号来说，对应的训练数据是已知的，上述处理器1039或者训练器1059以这些图像信号作为处理模型的输入数据，获取输出后的预测数据。上述处理器1039或者训练器1059计算上述预测数据与其对应的训练数据的相关系数，并根据该自适应地更新上述处理模型中的参数(如权值、偏置等等)，以提高上述处理模型的性能，进而得到上述目标处理模型。
在一种可行的实施例中，所述信号处理装置100的指令转换器1029可以用于对指令转换器1029中的信号指令转换模型进行自适应训练，以得到目标信号指令转换模型：所述信号处理装置100的处理器1039可以用于对处理器1039中的处理模型进行自适应训练，以得到目标处理模型。
在一种可行的实施例中，所述信号处理装置100还包括：
训练器105，用于对指令转换器1029中的信号指令转换模型和处理器1039中的处理模型分别进行自适应训练，以得到目标信号指令转换模型和目标处理模型。
上述训练器1059可以通过有监督的方法或者无监督的方法调整上述信号指令转换模型中或者上述处理模型中的结构与参数，以提高该信号指令转换模型或者处理模型的性能，最终得到目标信号指令转换模型或者目标处理模型。
在本实施例中，信号处理装置100是以模块的形式来呈现。这里的“模块”可以指特定应用集成电路(application-specific integrated circuit，ASIC)，执行一个或多个软件或固件程序的处理器和存储器，集成逻辑电路，和/或其他可以提供上述功能的器件。此外，以上信号采集器1019、指令转换器1029、处理器1039、存储模块104和训练器1059可通过图5A～图8所示的装置来实现。
可选地，上述信号处理装置100的指令转换器1029或者处理器1039的处理模块1032为人工神经网络芯片，即上述指令转换器1029和上述处理器1039的处理模块1032可以是两块独立的人工神经网 络芯片，其结构分别如图5A～图8所示。
在本装置中，指令转换器1029和处理器1039既可以串行执行，也可以通过软流水的方式执行，即处理器1039在处理上一个图像时，指令转换器1029可以处理下一个图像，这样可以提高硬件的吞吐率，提高图像处理效率。
参阅图5A，提供了一种计算装置，该计算装置用于执行机器学习计算，该计算装置包括：控制器单元11和运算单元12，其中，控制器单元11与运算单元12连接，该运算单元12包括：一个主处理电路和多个从处理电路；
控制器单元11，用于获取输入数据以及计算指令；在一种可选方案中，具体的，获取输入数据以及计算指令方式可以通过数据输入输出单元得到，该数据输入输出单元具体可以为一个或多个数据I/O接口或I/O引脚。
上述计算指令包括但不限于：正向运算指令或反向训练指令，或其他神经网络运算指令等等，例如卷积运算指令，本申请具体实施方式并不限制上述计算指令的具体表现形式。
控制器单元11，还用于解析该计算指令得到多个运算指令，将该多个运算指令以及所述输入数据发送给所述主处理电路；
主处理电路101，用于对所述输入数据执行前序处理以及与所述多个从处理电路之间传输数据以及运算指令；
多个从处理电路102，用于依据从所述主处理电路传输的数据以及运算指令并行执行中间运算得到多个中间结果，并将多个中间结果传输给所述主处理电路；
主处理电路101，用于对所述多个中间结果执行后续处理得到所述计算指令的计算结果。
本申请提供的技术方案将运算单元设置成一主多从结构，对于正向运算的计算指令，其可以将依据正向运算的计算指令将数据进行拆分，这样通过多个从处理电路即能够对计算量较大的部分进行并行运算，从而提高运算速度，节省运算时间，进而降低功耗。
可选的，上述机器学习计算具体可以包括：人工神经网络运算，上述输入数据具体可以包括：输入神经元数据和权值数据。上述计算结果具体可以为：人工神经网络运算的结果即输出神经元数据。
对于神经网络中的运算可以为神经网络中的一层的运算，对于多层神经网络，其实现过程是，在正向运算中，当上一层人工神经网络执行完成之后，下一层的运算指令会将运算单元中计算出的输出神经元作为下一层的输入神经元进行运算(或者是对该输出神经元进行某些操作再作为下一层的输入神经元)，同时，将权值也替换为下一层的权值；在反向运算中，当上一层人工神经网络的反向运算执行完成后，下一层运算指令会将运算单元中计算出的输入神经元梯度作为下一层的输出神经元梯度进行运算(或者是对该输入神经元梯度进行某些操作再作为下一层的输出神经元梯度)，同时将权值替换为下一层的权值。
上述机器学习计算还可以包括支持向量机运算，k-近邻(k-nn)运算，k-均值(k-means)运算，主成分分析运算等等。为了描述的方便，下面以人工神经网络运算为例来说明机器学习计算的具体方案。
对于人工神经网络运算，如果该人工神经网络运算具有多层运算，多层运算的输入神经元和输出神经元并非是指整个神经网络的输入层中神经元和输出层中神经元，而是对于网络中任意相邻的两层，处于网络正向运算下层中的神经元即为输入神经元，处于网络正向运算上层中的神经元即为输出神经元。以卷积神经网络为例，设一个卷积神经网络有L层，K＝1,2,...,L-1，对于第K层和第K+1层来说，我们将第K层称为输入层，其中的神经元为所述输入神经元，第K+1层称为输出层，其中的神经元为所述输出神经元。即除最顶层外，每一层都可以作为输入层，其下一层为对应的输出层。
可选的，上述计算装置还可以包括：该存储单元10和直接内存访问单元50，存储单元10可以包括：寄存器、缓存中的一个或任意组合，具体的，所述缓存，用于存储所述计算指令；所述寄存器，用于存储所述输入数据和标量；所述缓存为高速暂存缓存。直接内存访问单元50用于从存储单元10读取或存 储数据。
可选的，该控制器单元包括：指令存储单元110、指令处理单元111和存储队列单元113；
指令存储单元110，用于存储所述人工神经网络运算关联的计算指令；
所述指令处理单元111，用于对所述计算指令解析得到多个运算指令；
存储队列单元113，用于存储指令队列，该指令队列包括：按该队列的前后顺序待执行的多个运算指令或计算指令。
举例说明，在一个可选的技术方案中，主运算处理电路也可以包括一个控制器单元，该控制器单元可以包括主指令处理单元，具体用于将指令译码成微指令。当然在另一种可选方案中，从运算处理电路也可以包括另一个控制器单元，该另一个控制器单元包括从指令处理单元，具体用于接收并处理微指令。上述微指令可以为指令的下一级指令，该微指令可以通过对指令的拆分或解码后获得，能被进一步解码为各部件、各单元或各处理电路的控制信号。
在一种可选方案中，该计算指令的结构可以如下表所示。
操作码 寄存器或立即数 寄存器/立即数 ...
上表中的省略号表示可以包括多个寄存器或立即数。
在另一种可选方案中，该计算指令可以包括：一个或多个操作域以及一个操作码。该计算指令可以包括神经网络运算指令。以神经网络运算指令为例，如表1所示，其中，寄存器号0、寄存器号1、寄存器号2、寄存器号3、寄存器号4可以为操作域。其中，每个寄存器号0、寄存器号1、寄存器号2、寄存器号3、寄存器号4可以是一个或者多个寄存器的号码。
Figure PCTCN2018112528-appb-000001
上述寄存器可以为片外存储器，当然在实际应用中，也可以为片内存储器，用于存储数据，该数据具体可以为n维数据，n为大于等于1的整数，例如，n＝1时，为1维数据，即向量，如n＝2时，为2维数据，即矩阵，如n＝3或3以上时，为多维张量。
可选的，该控制器单元还可以包括：
所述依赖关系处理单元108，用于在具有多个运算指令时，确定第一运算指令与所述第一运算指令之前的第零运算指令是否存在关联关系，如所述第一运算指令与所述第零运算指令存在关联关系，则将所述第一运算指令缓存在所述指令存储单元内，在所述第零运算指令执行完毕后，从所述指令存储单元提取所述第一运算指令传输至所述运算单元；
所述确定该第一运算指令与第一运算指令之前的第零运算指令是否存在关联关系包括：
依据所述第一运算指令提取所述第一运算指令中所需数据(例如矩阵)的第一存储地址区间，依据所述第零运算指令提取所述第零运算指令中所需矩阵的第零存储地址区间，如所述第一存储地址区间与所述第零存储地址区间具有重叠的区域，则确定所述第一运算指令与所述第零运算指令具有关联关系，如所述第一存储地址区间与所述第零存储地址区间不具有重叠的区域，则确定所述第一运算指令与所述 第零运算指令不具有关联关系。
在另一种可选实施例中，运算单元12如图5C所示，可以包括一个主处理电路101和多个从处理电路102。在一个实施例里，如图5C所示，多个从处理电路呈阵列分布；每个从处理电路与相邻的其他从处理电路连接，主处理电路连接所述多个从处理电路中的k个从处理电路，所述k个从处理电路为：第1行的n个从处理电路、第m行的n个从处理电路以及第1列的m个从处理电路，需要说明的是，如图5C所示的K个从处理电路仅包括第1行的n个从处理电路、第m行的n个从处理电路以及第1列的m个从处理电路，即该k个从处理电路为多个从处理电路中直接与主处理电路连接的从处理电路。
K个从处理电路，用于在所述主处理电路以及多个从处理电路之间的数据以及指令的转发。
可选的，如图5D所示，该主处理电路还可以包括：转换处理电路110、激活处理电路111、加法处理电路112中的一种或任意组合；
转换处理电路110，用于将主处理电路接收的数据块或中间结果执行第一数据结构与第二数据结构之间的互换(例如连续数据与离散数据的转换)；或将主处理电路接收的数据块或中间结果执行第一数据类型与第二数据类型之间的互换(例如定点类型与浮点类型的转换)；
激活处理电路111，用于执行主处理电路内数据的激活运算；
加法处理电路112，用于执行加法运算或累加运算。
所述主处理电路，用于将确定所述输入神经元为广播数据，权值为分发数据，将分发数据分配成多个数据块，将所述多个数据块中的至少一个数据块以及多个运算指令中的至少一个运算指令发送给所述从处理电路；
所述多个从处理电路，用于依据该运算指令对接收到的数据块执行运算得到中间结果，并将运算结果传输给所述主处理电路；
所述主处理电路，用于将多个从处理电路发送的中间结果进行处理得到该计算指令的结果，将该计算指令的结果发送给所述控制器单元。
所述从处理电路包括：乘法处理电路；
所述乘法处理电路，用于对接收到的数据块执行乘积运算得到乘积结果；
转发处理电路(可选的)，用于将接收到的数据块或乘积结果转发。
累加处理电路，所述累加处理电路，用于对该乘积结果执行累加运算得到该中间结果。
另一个实施例里，该运算指令为矩阵乘以矩阵的指令、累加指令、激活指令等等计算指令。
下面通过神经网络运算指令来说明如图5A所示的计算装置的具体计算方法。对于神经网络运算指令来说，其实际需要执行的公式可以为:s＝s(∑wx i+b)，其中，即将权值w乘以输入数据x i，进行求和，然后加上偏置b后做激活运算s(h)，得到最终的输出结果s。
在一种可选的实施方案中，如图5E所示，所述运算单元包括：树型模块40，所述树型模块包括：一个根端口401和多个支端口404，所述树型模块的根端口连接所述主处理电路，所述树型模块的多个支端口分别连接多个从处理电路中的一个从处理电路；
上述树型模块具有收发功能，例如如图5E所示，该树型模块即为发送功能，如图6B所示，该树型模块即为接收功能。
所述树型模块，用于转发所述主处理电路与所述多个从处理电路之间的数据块、权值以及运算指令。
可选的，该树型模块为计算装置的可选择结果，其可以包括至少1层节点，该节点为具有转发功能的线结构，该节点本身可以不具有计算功能。如树型模块具有零层节点，即无需该树型模块。
可选的，该树型模块可以为n叉树结构，例如，如图5F所示的二叉树结构，当然也可以为三叉树结构，该n可以为大于等于2的整数。本申请具体实施方式并不限制上述n的具体取值，上述层数也可以为2，从处理电路可以连接除倒数第二层节点以外的其他层的节点，例如可以连接如图5F所示的倒数第一层的节点。
可选的，上述运算单元可以携带单独的缓存，如图5G所示，可以包括：神经元缓存单元，该神经 元缓存单元63缓存该从处理电路的输入神经元向量数据和输出神经元值数据。
如图5H所示，该运算单元还可以包括：权值缓存单元64，用于缓存该从处理电路在计算过程中需要的权值数据。
在一种可选实施例中，运算单元12如图5B所示，可以包括分支处理电路103；其具体的连接结构如图5B所示，其中，
主处理电路101与分支处理电路103(一个或多个)连接，分支处理电路103与一个或多个从处理电路102连接；
分支处理电路103，用于执行转发主处理电路101与从处理电路102之间的数据或指令。
在一种可选实施例中，以神经网络运算中的全连接运算为例，过程可以为：y＝f(wx+b)，其中，x为输入神经元矩阵，w为权值矩阵，b为偏置标量，f为激活函数，具体可以为：sigmoid函数，tanh、relu、softmax函数中的任意一个。这里假设为二叉树结构，具有8个从处理电路，其实现的方法可以为：
控制器单元从存储单元内获取输入神经元矩阵x，权值矩阵w以及全连接运算指令，将输入神经元矩阵x，权值矩阵w以及全连接运算指令传输给主处理电路；
主处理电路确定该输入神经元矩阵x为广播数据，确定权值矩阵w为分发数据，将权值矩阵w拆分成8个子矩阵，然后将8个子矩阵通过树型模块分发给8个从处理电路，将输入神经元矩阵x广播给8个从处理电路，
从处理电路并行执行8个子矩阵与输入神经元矩阵x的乘法运算和累加运算得到8个中间结果，将8个中间结果发送给主处理电路；
主处理电路，用于将8个中间结果排序得到wx的运算结果，将该运算结果执行偏置b的运算后执行激活操作得到最终结果y，将最终结果y发送至控制器单元，控制器单元将该最终结果y输出或存储至存储单元内。
如图5A所示的计算装置执行神经网络正向运算指令的方法具体可以为：
控制器单元从指令存储单元内提取神经网络正向运算指令、神经网络运算指令对应的操作域以及至少一个操作码，控制器单元将该操作域传输至数据访问单元，将该至少一个操作码发送至运算单元。
控制器单元从存储单元内提取该操作域对应的权值w和偏置b(当b为0时，不需要提取偏置b)，将权值w和偏置b传输至运算单元的主处理电路，控制器单元从存储单元内提取输入数据Xi，将该输入数据Xi发送至主处理电路。
主处理电路依据该至少一个操作码确定为乘法运算，确定输入数据Xi为广播数据，确定权值数据为分发数据，将权值w拆分成n个数据块；
控制器单元的指令处理单元依据该至少一个操作码确定乘法指令、偏置指令和累加指令，将乘法指令、偏置指令和累加指令发送至主处理电路，主处理电路将该乘法指令、输入数据Xi以广播的方式发送给多个从处理电路，将该n个数据块分发给该多个从处理电路(例如具有n个从处理电路，那么每个从处理电路发送一个数据块)；多个从处理电路，用于依据该乘法指令将该输入数据Xi与接收到的数据块执行乘法运算得到中间结果，将该中间结果发送至主处理电路，该主处理电路依据该累加指令将多个从处理电路发送的中间结果执行累加运算得到累加结果，依据该偏置指令将该累加结果执行加偏置b得到最终结果，将该最终结果发送至该控制器单元。
另外，加法运算和乘法运算的顺序可以调换。
本申请提供的技术方案通过一个指令即神经网络运算指令即实现了神经网络的乘法运算以及偏置运算，在神经网络计算的中间结果均无需存储或提取，减少了中间数据的存储以及提取操作，所以其具有减少对应的操作步骤，提高神经网络的计算效果的优点。
本申请还揭露了一个机器学习运算装置，其包括一个或多个在本申请中提到的计算装置，用于从其他处理装置中获取待运算数据和控制信息，执行指定的机器学习运算，执行结果通过I/O接口传递给外围设备。外围设备譬如摄像头，显示器，鼠标，键盘，网卡，wifi接口，服务器。当包含一个以上计算 装置时，计算装置间可以通过特定的结构进行链接并传输数据，譬如，通过PCIE总线进行互联并传输数据，以支持更大规模的机器学习的运算。此时，可以共享同一控制系统，也可以有各自独立的控制系统；可以共享内存，也可以每个加速器有各自的内存。此外，其互联方式可以是任意互联拓扑。
该机器学习运算装置具有较高的兼容性，可通过PCIE接口与各种类型的服务器相连接。
本申请还揭露了一个组合处理装置，其包括上述的机器学习运算装置，通用互联接口，和其他处理装置。机器学习运算装置与其他处理装置进行交互，共同完成用户指定的操作。图6A为组合处理装置的示意图。
其他处理装置，包括中央处理器CPU、图形处理器GPU、神经网络处理器等通用/专用处理器中的一种或以上的处理器类型。其他处理装置所包括的处理器数量不做限制。其他处理装置作为机器学习运算装置与外部数据和控制的接口，包括数据搬运，完成对本机器学习运算装置的开启、停止等基本控制；其他处理装置也可以和机器学习运算装置协作共同完成运算任务。
通用互联接口，用于在所述机器学习运算装置与其他处理装置间传输数据和控制指令。该机器学习运算装置从其他处理装置中获取所需的输入数据，写入机器学习运算装置片上的存储装置；可以从其他处理装置中获取控制指令，写入机器学习运算装置片上的控制缓存；也可以读取机器学习运算装置的存储模块中的数据并传输给其他处理装置。
可选的，该结构如图7A所示，还可以包括存储装置，存储装置分别与所述机器学习运算装置和所述其他处理装置连接。存储装置用于保存在所述机器学习运算装置和所述其他处理装置的数据，尤其适用于所需要运算的数据在本机器学习运算装置或其他处理装置的内部存储中无法全部保存的数据。
该组合处理装置可以作为手机、机器人、无人机、视频监控设备等设备的SOC片上系统，有效降低控制部分的核心面积，提高处理速度，降低整体功耗。此情况时，该组合处理装置的通用互联接口与设备的某些部件相连接。某些部件譬如摄像头，显示器，鼠标，键盘，网卡，wifi接口。
在一些实施例里，还申请了一种芯片，其包括了上述机器学习运算装置或组合处理装置。
在一些实施例里，申请了一种芯片封装结构，其包括了上述芯片。
在一些实施例里，申请了一种板卡，其包括了上述芯片封装结构。参阅图7B，图7B提供了一种板卡，上述板卡除了包括上述芯片389以外，还可以包括其他的配套部件，该配套部件包括但不限于：存储器件390、接口装置391和控制器件392；
所述存储器件390与所述芯片封装结构内的芯片通过总线连接，用于存储数据。所述存储器件可以包括多组存储单元393。每一组所述存储单元与所述芯片通过总线连接。可以理解，每一组所述存储单元可以是DDR SDRAM(英文：Double Data Rate SDRAM，双倍速率同步动态随机存储器)。
DDR不需要提高时钟频率就能加倍提高SDRAM的速度。DDR允许在时钟脉冲的上升沿和下降沿读出数据。DDR的速度是标准SDRAM的两倍。在一个实施例中，所述存储装置可以包括4组所述存储单元。每一组所述存储单元可以包括多个DDR4颗粒(芯片)。在一个实施例中，所述芯片内部可以包括4个72位DDR4控制器，上述72位DDR4控制器中64bit用于传输数据，8bit用于ECC校验。可以理解，当每一组所述存储单元中采用DDR4-3200颗粒时，数据传输的理论带宽可达到25600MB/s。
在一个实施例中，每一组所述存储单元包括多个并联设置的双倍速率同步动态随机存储器。DDR在一个时钟周期内可以传输两次数据。在所述芯片中设置控制DDR的控制器，用于对每个所述存储单元的数据传输与数据存储的控制。
所述接口装置与所述芯片封装结构内的芯片电连接。所述接口装置用于实现所述芯片与外部设备(例如服务器或计算机)之间的数据传输。例如在一个实施例中，所述接口装置可以为标准PCIE接口。比如，待处理的数据由服务器通过标准PCIE接口传递至所述芯片，实现数据转移。可选的，当采用PCIE3.0X 16接口传输时，理论带宽可达到16000MB/s。在另一个实施例中，所述接口装置还可以是其他的接口，本申请并不限制上述其他的接口的具体表现形式，所述接口单元能够实现转接功能即可。另外， 所述芯片的计算结果仍由所述接口装置传送回外部设备(例如服务器)。
所述控制器件与所述芯片电连接。所述控制器件用于对所述芯片的状态进行监控。具体的，所述芯片与所述控制器件可以通过SPI接口电连接。所述控制器件可以包括单片机(Micro Controller Unit，MCU)。如所述芯片可以包括多个处理芯片、多个处理核或多个处理电路，可以带动多个负载。因此，所述芯片可以处于多负载和轻负载等不同的工作状态。通过所述控制装置可以实现对所述芯片中多个处理芯片、多个处理和或多个处理电路的工作状态的调控。
在一些实施例里，申请了一种电子设备，其包括了上述板卡。
电子设备包括数据处理装置、机器人、电脑、打印机、扫描仪、平板电脑、智能终端、手机、行车记录仪、导航仪、传感器、摄像头、服务器、云端服务器、相机、摄像机、投影仪、手表、耳机、移动存储、可穿戴设备、交通工具、家用电器、和/或医疗设备。
所述交通工具包括飞机、轮船和/或车辆；所述家用电器包括电视、空调、微波炉、冰箱、电饭煲、加湿器、洗衣机、电灯、燃气灶、油烟机；所述医疗设备包括核磁共振仪、B超仪和/或心电图仪。
如图8所示，信号采集器用于接收信号，并将信号和待处理数据作为输入数据传递给板卡内部的芯片。
可选地，上述图像处理芯片可为人工神经网络处理芯片。
可选地，信号采集器为麦克风或者多阵列麦克风。
其中板卡内部的芯片的包括同上述所示的实施例，用于得到对应的输出数据(即处理后的图像)，并将其传输至交互界面中。
其中交互界面接收上述芯片(可以看成人工神经网络处理器)的输出数据，并将其转化为合适形式的反馈信息显示给用户。
其中控制器单元接收用户的操作或命令，并控制整个信号处理装置的运作。
可选地，上述电子设备可为数据处理装置、机器人、电脑、平板电脑、智能终端、手机、云端服务器、相机、摄像机、投影仪、手表、耳机、移动存储或者可穿戴设备。
参见图9，图9为本发明实施例提供的一种信号处理方法的流程示意图。如图9所示，该方法包括：
901、信号处理装置采集用户输入的信号。
902、所述信号处理装置根据目标信号指令转换模型将所述信号转换成处理指令和目标数据，所述目标数据为待处理数据的处理数据。
在一种可行的实施例中，所述根据目标信号指令转换模型将所述信号转换成处理指令和目标数据，包括：
将所述信号通过语音识别技术、图像识别技术、自然语言处理技术等转换成文本信息；
将所述文本信息通过自然语言处理技术和所述目标信号指令转换模型转换成所述处理指令；
用于根据所述处理指令中的语义区域的粒度和图像识别技术对所述待处理数据进行区域划分，得到所述目标数据。
在一种可行的实施例中，所述根据目标信号指令转换模型将所述信号转换成处理指令和目标数据，包括：
将所述信号通过语音识别技术、语义理解技术、图像识别技术、自然语言处理技术和所述信号指令转换模型转换成所述处理指令；
根据所述处理指令中的语义区域的粒度和图像识别技术对所述待处理数据进行区域划分，得到所述目标数据。
在一种可行的实施例中，所述方法还包括：
存储所述文本信息或者所述处理指令或者所述目标数据。
903、所述信号处理装置根据所述处理指令和目标处理模型对所述目标数据进行处理。
在一种可行的实施例中，所述根据所述处理指令和目标处理模型对所述目标数据进行处理，包括：
在预设时间窗口内从所述存储模块中获取M条处理指令，所述M为大于1的整数；
根据所述M条处理指令和所述目标处理模型对所述目标数据进行处理。
在一种可行的实施例中，所述根据所述M条处理指令和所述目标处理模型对所述目标数据进行处理，包括：
删除所述M条处理指令中，功能相同的处理指令，以得到N条处理指令，所述N为小于所述M的整数；
根据所述N条处理指令和所述目标处理模型对所述目标数据进行处理。
在一种可行的实施例中，所述接收信号和待处理数据之前，所述方法还包括：
对信号指令转换模型进行自适应训练，以得到目标信号指令转换模型。
在一种可行的实施例中，所述对所述信号指令转换模型进行自适应训练是离线进行的或者是离线进行的。
在一种可行的实施例中，所述对所述信号指令转换模型进行自适应训练是有监督的或者是无监督的。
在一种可行的实施例中，所述对信号指令转换模型进行自适应训练，以得到目标信号指令转换模型，包括：
根据所述信号指令转换模型将所述信号换成预测指令；
确定所述预测指令与其对应的指令集合的相关系数；
根据所述预测指令与其对应的指令集合的相关系数优化所述信号指令转换模型，以得到所述目标信号指令转换模型。
在一种可行的实施例中，所述采集用户输入的信号之前，所述方法还包括：
对处理模型进行自适应训练，以得到目标处理模型。
在一种可行的实施例中，所述对所述处理模型进行自适应训练是离线进行的或者是离线进行的。
在一种可行的实施例中，所述对所述处理模型进行自适应训练是有监督或者无监督的。
在一种可行的实施例中，所述对处理模型进行自适应训练，以得到目标处理模型，包括：
根据所述处理模型对所述待处理数据进行处理，以得到预测数据；
确定所述预测数据与其对应的训练数据的相关系数；
根据所述预测数据与其对应的训练数据的相关系数优化所述处理模型，以得到所述目标处理模型。
在一种可行的实施例中，所述方法还包括：
发送交互信息给用户，接收用户反馈信息，并依据该反馈信息生成相应的指令。
需要说明的是，图9所示的方法的各个步骤的具体实现过程可参见上述信号处理装置的具体实现过程，在此不再叙述。
本发明实施例还提供一种计算机存储介质，其中，该计算机存储介质存储用于电子数据交换的计算机程序，该计算机程序使得计算机执行如上述方法实施例中记载的任一方法的部分或全部步骤，上述计算机包括电子设备。
本发明实施例还提供一种计算机程序产品，上述计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质，上述计算机程序可操作来使计算机执行如上述方法实施例中记载的任一方法的部分或全部步骤。该计算机程序产品可以为一个软件安装包，上述计算机包括电子设备。
需要说明的是，对于前述的各方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本发明并不受所描述的动作顺序的限制，因为依据本发明，某些步骤可以采用其他顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于可选 实施例，所涉及的动作和模块并不一定是本发明所必须的。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其他实施例的相关描述。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置，可通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元可以采用硬件的形式实现。
以上对本发明实施例进行了详细介绍，本文中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想；同时，对于本领域的一般技术人员，依据本发明的思想，在具体实施方式及应用范围上均会有改变之处，综上上述，本说明书内容不应理解为对本发明的限制。

Claims (30)

  1. 一种信号处理装置，其特征在于，包括：
    信号采集器，用于采集用户输入的信号；
    指令转换器，用于根据目标信号指令转换模型将所述信号转换成处理指令和目标数据，所述目标数据为待处理数据的处理数据；
    处理器，用于根据所述处理指令和目标处理模型对所述目标数据进行处理。
  2. 根据权利要求1所述的信号处理装置，其特征在于，所述指令转换器包括：
    第一信号识别器，用于将信号通过信号识别技术转换成文本信息，所述信号识别技术为以下至少一种：语音识别技术、语义理解技术、图像识别技术、自然语言处理技术；
    信号文本转换器，用于将文本信息通过自然语言处理技术和所述目标信号指令转换模型转换成所述处理指令；
    第一数据识别器，用于根据所述处理指令中的语义区域的粒度和识别技术对所述待处理数据进行划分，获取所述目标数据。
  3. 根据权利要求1所述的信号处理装置，其特征在于，所述指令转换器包括：
    第二信号识别器，用于将所述信号通过所述信号识别技术和所述目标信号指令转换模型转换成所述处理指令；
    第二数据识别器，用于根据所述处理指令中的语义区域的粒度和识别技术对所述待处理数据进行划分，得到所述目标数据。
  4. 根据权利要求1-3任一项所述的信号处理装置，其特征在于，所述信号处理装置还包括：
    存储器，用于存储所述文本信息或者所述处理指令或者所述目标数据。
  5. 根据权利要求4所述的信号处理装置，其特征在于，所述处理器包括：
    取指令模块，用于在预设时间窗口内从所述存储器中获取M条处理指令；
    处理模块，用于根据所述M条处理指令和所述目标处理模型对所述目标数据进行处理。
  6. 根据权利要求5所述的信号处理装置，其特征在于，所述处理模块用于：
    删除所述M条处理指令中功能相同的处理指令，得到N条处理指令，所述M为大于1的整数，所述N为小于所述M的整数；
    根据所述N条处理指令和所述目标处理模型对所述目标数据进行处理。
  7. 根据权利要求1的信号处理装置，其特征在于，所述指令转换器用于：
    对信号指令转换模型进行自适应训练，以得到所述目标信号指令转换模型。
  8. 根据权利要求7所述的信号处理装置，其特征在于，所述指令转换器还用于：
    根据所述信号指令转换模型将所述信号换成预测指令；
    确定所述预测指令与其对应的指令集合的相关系数；
    根据所述预测指令与其对应的指令集合的相关系数优化所述信号指令转换模型，以得到所述目标信号指令转换模型。
  9. 根据权利要求1所述的信号处理装置，其特征在于，所述信号处理装置还包括：
    训练器，用于根据所述信号指令转换模型将所述信号换成预测指令；确定所述预测指令与其对应的指令集合的相关系数；根据所述预测指令与其对应的指令集合的相关系数优化所述信号指令转换模型，以得到所述目标信号指令转换模型。
  10. 根据权利要求1所述的信号处理装置，其特征在于，所述处理器用于：
    对处理模型进行自适应训练，以得到所述目标处理模型。
  11. 根据权利要求10所述的信号处理装置，其特征在于，所述处理器还用于：
    根据所述处理模型对所述待处理数据进行处理，以得到预测数据；
    确定所述预测数据与其对应的训练数据的相关系数；
    根据所述预测数据与其对应的训练数据的相关系数优化所述处理模型，以得到所述目标处理模型。
  12. 根据权利要求1所述的信号处理装置，其特征在于，所述训练器还用于：
    根据所述处理模型对所述待处理数据进行处理，以得到预测数据；
    确定所述预测数据与其对应的训练数据的相关系数；
    根据所述预测数据与其对应的训练数据的相关系数优化所述处理模型，以得到所述目标处理模型。
  13. 根据权利要求1所述的信号处理装置，其特征在于，所述采集用户输入的信号之前，所述信号处理装置还用于：
    对信号指令转换模型进行自适应训练，以得到所述目标信号指令转换模型。
  14. 根据权利要求13所述的信号处理装置，其特征在于，所述信号处理装置具体用于
    根据所述信号指令转换模型将所述语音信号换成预测指令；
    确定所述预测指令与其对应的指令集合的相关系数；
    根据所述预测指令与其对应的指令集合的相关系数优化所述信号指令转换模型，以得到所述目标信号指令转换模型。
  15. 根据权利要求1所述的信号处理装置，其特征在于，所述信号处理装置还用于：
    对处理模型进行自适应训练，以得到所述目标处理模型。
  16. 根据权利要求15所述的信号处理装置，其特征在于，所述信号处理装置具体用于：
    根据所述处理模型对所述待处理数据进行处理，以得到预测数据；
    确定所述预测数据与其对应的训练数据的相关系数；
    根据所述预测数据与其对应的训练数据的相关系数优化所述处理模型，以得到所述目标处理模型。
  17. 根据权利要求1-16任一项所述的信号处理装置，其特征在于，所述指令转换器还包括：
    交互模块，用于发送交互信息给用户，接收用户反馈信息，并依据该反馈信息生成相应的指令。
  18. 一种机器学习运算装置，其特征在于，所述机器学习运算装置包括一个或多个如权利要求1-17任一项所述的信号处理装置，用于从其他处理装置中获取待运算输入数据和控制信息，并执行指定的机器学习运算，将执行结果通过I/O接口传递给其他处理装置；
    当所述机器学习运算装置包含多个所述信号处理装置时，所述多个所述信号处理装置间可以通过特定的结构进行连接并传输数据；
    其中，多个所述信号处理装置通过快速外部设备互连总线PCIE总线进行互联并传输数据，以支持更大规模的机器学习的运算；多个所述信号处理装置共享同一控制系统或拥有各自的控制系统；多个所述信号处理装置共享内存或者拥有各自的内存；多个所述信号处理装置的互联方式是任意互联拓扑。
  19. 一种组合处理装置，其特征在于，所述组合处理装置包括如权利要求18所述的机器学习运算装置，通用互联接口和其他处理装置；
    所述机器学习运算装置与所述其他处理装置进行交互，共同完成用户指定的计算操作。
  20. 根据权利要求19所述的组合处理装置，其特征在于，还包括：存储装置，该存储装置分别与所述机器学习运算装置和所述其他处理装置连接，用于保存所述机器学习运算装置和所述其他处理装置的数据。
  21. 一种神经网络芯片，其特征在于，所述机器学习芯片包括如权利要求18所述的机器学习运算装置或如权利要求19所述的组合处理装置或如权利要求20所述的组合处理装置。
  22. 一种电子设备，其特征在于，所述电子设备包括如所述权利要求20所述的芯片。
  23. 一种板卡，其特征在于，所述板卡包括：存储器件、接口装置和控制器件以及如权利要求21所述的神经网络芯片；
    其中，所述神经网络芯片与所述存储器件、所述控制器件以及所述接口装置分别连接；
    所述存储器件，用于存储数据；
    所述接口装置，用于实现所述芯片与外部设备之间的数据传输；
    所述控制器件，用于对所述芯片的状态进行监控。
  24. 一种信号处理方法，其特征在于，包括：
    采集用户输入的信号；
    根据目标信号指令转换模型将所述信号转换成处理指令和目标数据，所述目标数据为待处理数据的部分数据；
    根据所述处理指令和目标处理模型对所述目标数据进行处理。
  25. 根据权利要求24所述的信号处理方法，其特征在于，所述根据目标信号指令转换模型将所述信号转换成处理指令和处理数据，包括：
    将信号通过信号识别技术转换成文本信息，所述信号识别技术为以下至少一种：语音识别技术、语义理解技术、图像识别技术、自然语言处理技术；
    将文本信息通过自然语言处理技术和所述目标信号指令转换模型转换成所述处理指令；
    根据所述处理指令中的语义区域的粒度和图像识别技术对所述待处理数据进行区域划分，得到所述目标数据。
  26. 根据权利要求24所述的信号处理方法，其特征在于，所述根据目标信号指令转换模型将所述信号转换成处理指令和目标数据，包括：
    将所述信号通过所述信号识别技术和所述目标信号指令转换模型转换成所述处理指令；
    根据所述处理指令中的语义区域的粒度和图像识别技术对所述待处理数据进行区域划分，得到所述目标数据。
  27. 根据权利要求24-26任一项所述的信号处理方法，其特征在于，所述方法还包括：
    存储所述文本信息或者所述处理指令或者所述目标数据。
  28. 根据权利要求27所述的信号处理方法，其特征在于，所述根据所述处理指令和目标处理模型对所述目标数据进行处理，包括：
    在预设时间窗口内从所述存储器中获取M条处理指令；
    根据所述M条处理指令和所述目标处理模型对所述目标数据进行处理。
  29. 根据权利要求28所述的信号处理方法，其特征在于，所述根据所述M条处理指令和所述目标处理模型对所述目标数据进行处理，包括：
    删除所述M条处理指令中功能相同的处理指令，得到N条处理指令，所述M为大于1的整数，所述N为小于所述M的整数；
    根据所述N条处理指令和所述目标处理模型对所述目标数据进行处理。
  30. 根据权利要求24的信号处理方法，其特征在于，所述方法还包括：
    对信号指令转换模型进行自适应训练，以得到所述目标信号指令转换模型。
PCT/CN2018/112528 2018-09-28 2018-10-30 信号处理装置、信号处理方法及相关产品 WO2020062392A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP18920196.5A EP3859488A4 (en) 2018-09-28 2018-10-30 Signal processing device, signal processing method and related product
US16/620,540 US11703939B2 (en) 2018-09-28 2018-10-30 Signal processing device and related products

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
CN201811145439.8 2018-09-28
CN201811152999.6 2018-09-28
CN201811145756.XA CN110968285A (zh) 2018-09-28 2018-09-28 信号处理装置及相关产品
CN201811145439.8A CN110968235B (zh) 2018-09-28 2018-09-28 信号处理装置及相关产品
CN201811152999.6A CN110969246A (zh) 2018-09-28 2018-09-28 信号处理装置及相关产品
CN201811145756.X 2018-09-28

Publications (1)

Publication Number Publication Date
WO2020062392A1 true WO2020062392A1 (zh) 2020-04-02

Family

ID=69949533

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/112528 WO2020062392A1 (zh) 2018-09-28 2018-10-30 信号处理装置、信号处理方法及相关产品

Country Status (3)

Country Link
US (1) US11703939B2 (zh)
EP (1) EP3859488A4 (zh)
WO (1) WO2020062392A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111680527A (zh) * 2020-06-09 2020-09-18 语联网（武汉）信息技术有限公司 基于专属机翻引擎训练的人机共译系统与方法
US20230386470A1 (en) * 2021-01-06 2023-11-30 Boe Technology Group Co., Ltd. Speech instruction recognition method, electronic device, and non-transient computer readable storage medium

Families Citing this family (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11437032B2 (en) 2017-09-29 2022-09-06 Shanghai Cambricon Information Technology Co., Ltd Image processing apparatus and method
US11630666B2 (en) 2018-02-13 2023-04-18 Shanghai Cambricon Information Technology Co., Ltd Computing device and method
US20200159532A1 (en) 2018-02-13 2020-05-21 Shanghai Cambricon Information Technology Co., Ltd Computing device and method
KR102354720B1 (ko) 2018-02-13 2022-01-21 상하이 캠브리콘 인포메이션 테크놀로지 컴퍼니 리미티드 계산 장치 및 방법
CN110162162B (zh) 2018-02-14 2023-08-18 上海寒武纪信息科技有限公司 处理器的控制装置、方法及设备
EP3624020A4 (en) 2018-05-18 2021-05-05 Shanghai Cambricon Information Technology Co., Ltd CALCULATION PROCEDURES AND RELATED PRODUCTS
KR102470893B1 (ko) 2018-06-27 2022-11-25 상하이 캠브리콘 인포메이션 테크놀로지 컴퍼니 리미티드 온 칩 코드의 브레이크 포인트에 의한 디버그 방법, 온 칩 프로세서 및 브레이크 포인트에 의한 칩 디버그 시스템
EP3640810A4 (en) 2018-08-28 2021-05-05 Cambricon Technologies Corporation Limited DATA PRE-PROCESSING PROCESS AND APPARATUS, COMPUTER DEVICE AND STORAGE MEDIA
CN110569864A (zh) * 2018-09-04 2019-12-13 阿里巴巴集团控股有限公司 基于gan网络的车损图像生成方法和装置
US11703939B2 (en) 2018-09-28 2023-07-18 Shanghai Cambricon Information Technology Co., Ltd Signal processing device and related products
CN111383638A (zh) 2018-12-28 2020-07-07 上海寒武纪信息科技有限公司 信号处理装置、信号处理方法及相关产品
US20200334522A1 (en) 2019-04-18 2020-10-22 Cambricon Technologies Corporation Limited Data processing method and related products
CN111832737B (zh) 2019-04-18 2024-01-09 中科寒武纪科技股份有限公司 一种数据处理方法及相关产品
CN112085193B (zh) 2019-06-12 2024-03-29 上海寒武纪信息科技有限公司 一种神经网络的量化参数确定方法及相关产品
US11676028B2 (en) 2019-06-12 2023-06-13 Shanghai Cambricon Information Technology Co., Ltd Neural network quantization parameter determination method and related products
CN110827852B (zh) * 2019-11-13 2022-03-04 腾讯音乐娱乐科技（深圳）有限公司 一种有效语音信号的检测方法、装置及设备
US11562518B2 (en) * 2020-06-05 2023-01-24 Google Llc Image manipulation by text instruction
KR20230124027A (ko) * 2020-12-18 2023-08-24 아지타 랩스, 인크. 격리 암호화를 통한 프라이버시 강화 컴퓨팅
US20220335087A1 (en) * 2021-04-14 2022-10-20 Ricoh Company, Ltd. Data processing apparatus, data processing system, and data processing method
CN115326656B (zh) * 2022-10-14 2022-12-23 西南交通大学 交通土建用颗粒材料松铺层颗粒粒径及级配无损测量方法
CN117711413A (zh) * 2023-11-02 2024-03-15 广东广信通信服务有限公司 一种语音识别数据处理方法、系统、装置及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105893419A (zh) * 2015-11-30 2016-08-24 乐视致新电子科技（天津）有限公司 一种多媒体照片生成方法、装置、设备及手机
CN106557165A (zh) * 2016-11-14 2017-04-05 北京智能管家科技有限公司 智能设备的动作模拟交互方法和装置及智能设备
CN106910503A (zh) * 2017-04-26 2017-06-30 海信集团有限公司 用于智能终端显示用户操控指令的方法、装置和智能终端
CN106997236A (zh) * 2016-01-25 2017-08-01 亮风台（上海）信息科技有限公司 基于多模态输入进行交互的方法和设备
US20180190257A1 (en) * 2016-12-29 2018-07-05 Shadecraft, Inc. Intelligent Umbrellas and/or Robotic Shading Systems Including Noise Cancellation or Reduction
CN108389098A (zh) * 2017-02-03 2018-08-10 北京京东尚科信息技术有限公司 语音购物方法以及系统
CN108509619A (zh) * 2018-04-04 2018-09-07 科大讯飞股份有限公司 一种语音交互方法及设备

Family Cites Families (194)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0375860A (ja) 1989-08-18 1991-03-29 Hitachi Ltd パーソナライズド端末
US5052043A (en) 1990-05-07 1991-09-24 Eastman Kodak Company Neural network with back propagation controlled through an output confidence measure
US6144977A (en) 1995-07-10 2000-11-07 Motorola, Inc. Circuit and method of converting a floating point number to a programmable fixed point number
GB9602701D0 (en) 1996-02-09 1996-04-10 Canon Kk Image manipulation
US7242414B1 (en) 1999-07-30 2007-07-10 Mips Technologies, Inc. Processor having a compare extension of an instruction set architecture
JP2000293371A (ja) 1999-04-09 2000-10-20 Hitachi Ltd マイクロプログラム制御方法及び装置
US6671796B1 (en) 2000-02-25 2003-12-30 Sun Microsystems, Inc. Converting an arbitrary fixed point value to a floating point value
US6931639B1 (en) 2000-08-24 2005-08-16 International Business Machines Corporation Method for implementing a variable-partitioned queue for simultaneous multithreaded processors
US7153348B2 (en) 2000-09-07 2006-12-26 Nippon Steel Corporation Hexavalent chromium-free surface-treating agent for Sn or Al-based coated steel sheet, and surface treated steel sheet
US20020138714A1 (en) 2001-03-22 2002-09-26 Sun Microsystems, Inc. Scoreboard for scheduling of instructions in a microprocessor that provides out of order execution
CN1270273C (zh) 2001-04-19 2006-08-16 艾利森电话股份有限公司 自适应存储器分配
US20030167460A1 (en) 2002-02-26 2003-09-04 Desai Vipul Anil Processor instruction set simulation power estimation method
US7236995B2 (en) 2002-12-27 2007-06-26 Arm Limited Data processing apparatus and method for converting a number between fixed-point and floating-point representations
DE10316381A1 (de) 2003-04-10 2004-10-28 Bayer Technology Services Gmbh Verfahren zum Training von neuronalen Netzen
JP4202244B2 (ja) 2003-12-22 2008-12-24 Ｎｅｃエレクトロニクス株式会社 Ｖｌｉｗ型ｄｓｐ，及びその動作方法
US20060161375A1 (en) 2004-12-30 2006-07-20 Allen Duberstein Optimizing processing speed based on measured temperatures
US7721128B2 (en) 2005-11-29 2010-05-18 International Business Machines Corporation Implementation of thermal throttling logic
CN1851668A (zh) 2006-06-01 2006-10-25 北京天碁科技有限公司 片上系统芯片、片上系统芯片的跟踪调试系统及方法
DE102006059156B4 (de) 2006-12-14 2008-11-06 Advanced Micro Devices, Inc., Sunnyvale Verfahren zum Testen eines integrierten Schaltkreischips mit zumindest zwei Schaltungskernen sowie integrierter Schaltkreischip und Testsystem
US20110060587A1 (en) 2007-03-07 2011-03-10 Phillips Michael S Command and control utilizing ancillary information in a mobile voice-to-speech application
US8560591B2 (en) 2007-04-25 2013-10-15 International Business Machines Corporation Detection of potential need to use a larger data format in performing floating point operations
US8051117B2 (en) 2007-04-26 2011-11-01 International Business Machines Corporation Shift significand of decimal floating point data
US8051118B2 (en) 2007-04-26 2011-11-01 International Business Machines Corporation Composition of decimal floating point data
US8190664B2 (en) 2007-04-26 2012-05-29 International Business Machines Corporation Employing a mask field of an instruction to encode a sign of a result of the instruction
JP5184824B2 (ja) 2007-06-15 2013-04-17 キヤノン株式会社 演算処理装置及び方法
JP2009110353A (ja) 2007-10-31 2009-05-21 Hitachi Ltd マイクロコントローラ及び制御システム
US7904287B2 (en) 2007-11-13 2011-03-08 International Business Machines Corporation Method and system for real-time prediction of power usage for a change to another performance state
JP4998794B2 (ja) 2007-11-29 2012-08-15 Ｎｋワークス株式会社 画像補正方法と画像補正装置
US20100073068A1 (en) 2008-09-22 2010-03-25 Hanwoo Cho Functional block level thermal control
JP5530729B2 (ja) * 2009-01-23 2014-06-25 本田技研工業株式会社 音声理解装置
CN101572829B (zh) 2009-06-10 2011-02-02 中国联合网络通信集团有限公司 Iptv视频质量监测方法、装置和系统
EP2336882A1 (en) 2009-12-18 2011-06-22 Telefonaktiebolaget L M Ericsson (PUBL) Technique for run-time provision of executable code using off-device services
CN102985673B (zh) 2010-04-21 2015-06-17 丰田自动车株式会社 内燃机的控制装置
JP2011253374A (ja) 2010-06-02 2011-12-15 Sony Corp 情報処理装置、および情報処理方法、並びにプログラム
US8452463B2 (en) 2010-06-04 2013-05-28 Apple Inc. Adjusting the thermal behavior of a computing system using indirect information about ambient temperature
US8694572B2 (en) 2010-07-06 2014-04-08 Silminds, Llc, Egypt Decimal floating-point fused multiply-add unit
US8924455B1 (en) 2011-02-25 2014-12-30 Xilinx, Inc. Multiplication of matrices using systolic arrays
CN102761509B (zh) 2011-04-27 2016-01-06 联芯科技有限公司 Ofdm系统的接收系统及降低接收系统内存的方法
CN103534664B (zh) 2011-05-12 2016-08-31 苹果公司 存在感测
CN102789413B (zh) 2011-05-23 2016-02-17 同济大学 一种并行程序的调试系统及方法
US8594982B2 (en) 2011-06-09 2013-11-26 Pulsar Informatics, Inc. Systems and methods for distributed calculation of fatigue-risk prediction and optimization
CN102404673B (zh) 2011-11-24 2013-12-18 苏州上声电子有限公司 数字化扬声器系统通道均衡与声场控制方法和装置
CN103152673B (zh) 2011-12-07 2015-07-08 中国科学院声学研究所 基于四元码动态失配整形的数字扬声器驱动方法和装置
CN102684701B (zh) 2012-04-27 2014-07-09 苏州上声电子有限公司 基于编码转换的数字扬声器驱动方法和装置
DE102012009502A1 (de) 2012-05-14 2013-11-14 Kisters Ag Verfahren zum Trainieren eines künstlichen neuronalen Netzes
US9417891B2 (en) 2012-06-11 2016-08-16 Vmware, Inc. Unified storage/VDI provisioning methodology
US9063731B2 (en) 2012-08-27 2015-06-23 Samsung Electronics Co., Ltd. Ultra low power apparatus and method to wake up a main processor
CN102903089B (zh) 2012-09-07 2014-12-17 山东大学 一种Linux环境下生成遥感图像快视图的方法
US9412366B2 (en) 2012-09-18 2016-08-09 Adobe Systems Incorporated Natural language image spatial and tonal localization
CN102981854A (zh) 2012-11-16 2013-03-20 天津市天祥世联网络科技有限公司 基于浮点数运算内联函数库的神经网络优化方法
US20150369963A1 (en) 2012-11-22 2015-12-24 Keio University Acrylic copolymer, optical film, polarizing plate and liquid crystal display device
US9851977B2 (en) 2012-12-06 2017-12-26 Kalray Apparatus and method for combining thread warps with compatible execution masks for simultaneous execution and increased lane utilization
US9720732B1 (en) 2013-02-11 2017-08-01 Amazon Technologies, Inc. Parameter selection for optimization of task execution based on execution history for prior tasks
JP2014170295A (ja) 2013-03-01 2014-09-18 Honda Motor Co Ltd 物体認識システム及び物体認識方法
US20190138372A1 (en) 2013-04-29 2019-05-09 Moogsoft, Inc. System for managing an instructure with security
JP5963957B2 (ja) 2013-06-12 2016-08-03 三菱電機株式会社 開発環境システム、開発環境装置、開発環境提供方法及びプログラム
JP6184891B2 (ja) 2014-03-12 2017-08-23 東芝メモリ株式会社 情報処理装置、半導体チップ、情報処理方法およびプログラム
EP3001333A4 (en) * 2014-05-15 2016-08-24 Huawei Tech Co Ltd METHOD AND APPARATUS FOR SEARCHING OBJECTS
US9507405B2 (en) 2014-06-18 2016-11-29 Oracle International Corporation System and method for managing power in a chip multiprocessor using a proportional feedback mechanism
US9575537B2 (en) 2014-07-25 2017-02-21 Intel Corporation Adaptive algorithm for thermal throttling of multi-core processors with non-homogeneous performance states
US10282100B2 (en) 2014-08-19 2019-05-07 Samsung Electronics Co., Ltd. Data management scheme in virtualized hyperscale environments
GB2524126B (en) 2014-08-28 2016-07-27 Imagination Tech Ltd Combining paths
US10317992B2 (en) 2014-09-25 2019-06-11 Microsoft Technology Licensing, Llc Eye gaze for spoken language understanding in multi-modal conversational interactions
US9916130B2 (en) 2014-11-03 2018-03-13 Arm Limited Apparatus and method for vector processing
FR3030077B1 (fr) 2014-12-10 2016-12-02 Arnault Ioualalen Procede d'ajustement de la precision d'un programme d'ordinateur manipulant au moins un nombre a virgule.
KR20160071732A (ko) * 2014-12-12 2016-06-22 삼성전자주식회사 음성 입력을 처리하는 방법 및 장치
EP3035204B1 (en) 2014-12-19 2018-08-15 Intel Corporation Storage device and method for performing convolution operations
US20170061279A1 (en) 2015-01-14 2017-03-02 Intel Corporation Updating an artificial neural network using flexible fixed point representation
US20160328645A1 (en) 2015-05-08 2016-11-10 Qualcomm Incorporated Reduced computational complexity for fixed point neural network
US10083395B2 (en) 2015-05-21 2018-09-25 Google Llc Batch processing in a neural network processor
CN104899641B (zh) 2015-05-25 2018-07-13 杭州朗和科技有限公司 深度神经网络学习方法、处理器和深度神经网络学习系统
CN115100017A (zh) 2015-06-10 2022-09-23 无比视视觉技术有限公司 用于处理图像的图像处理器和方法
CN104978303B (zh) 2015-06-19 2019-06-04 上海兆芯集成电路有限公司 单芯片整合的传感器集线器和多传感器管理方法
CN106469291A (zh) 2015-08-19 2017-03-01 中兴通讯股份有限公司 图像处理方法及终端
US10664757B2 (en) * 2015-09-16 2020-05-26 International Business Machines Corporation Cognitive operations based on empirically constructed knowledge graphs
US10031765B2 (en) 2015-09-24 2018-07-24 Intel Corporation Instruction and logic for programmable fabric hierarchy and cache
WO2017055609A1 (en) 2015-09-30 2017-04-06 Piksel, Inc Improved video stream delivery via adaptive quality enhancement using error correction models
US11061672B2 (en) 2015-10-02 2021-07-13 Via Alliance Semiconductor Co., Ltd. Chained split execution of fused compound arithmetic operations
CN106570559A (zh) 2015-10-09 2017-04-19 阿里巴巴集团控股有限公司 一种基于神经网络的数据处理方法和装置
US9930248B2 (en) 2015-11-17 2018-03-27 Eman Bayani Digital image capturing device system and method
CN106814639A (zh) 2015-11-27 2017-06-09 富泰华工业（深圳）有限公司 语音控制系统及方法
US9898847B2 (en) * 2015-11-30 2018-02-20 Shanghai Sunson Activated Carbon Technology Co., Ltd. Multimedia picture generating method, device and electronic device
US10699186B2 (en) 2015-12-02 2020-06-30 Google Llc Determining orders of execution of a neural network
CN110135581B (zh) 2016-01-20 2020-11-06 中科寒武纪科技股份有限公司 用于执行人工神经网络反向运算的装置和方法
US10664766B2 (en) 2016-01-27 2020-05-26 Bonsai AI, Inc. Graphical user interface to an artificial intelligence engine utilized to generate one or more trained artificial intelligence models
US10497089B2 (en) 2016-01-29 2019-12-03 Fotonation Limited Convolutional neural network
EP3416105A4 (en) 2016-02-12 2019-02-20 Sony Corporation INFORMATION PROCESSING METHOD AND INFORMATION PROCESSING DEVICE
JP2017156511A (ja) 2016-03-01 2017-09-07 ソニー株式会社 情報処理装置、情報処理方法、およびプログラム
US10103714B2 (en) 2016-03-01 2018-10-16 Qualcomm Incorporated Adjust voltage for thermal mitigation
US10019779B2 (en) 2016-03-08 2018-07-10 Amazon Technologies, Inc. Browsing interface for item counterparts having different scales and lengths
CN109934331B (zh) 2016-04-29 2020-06-19 中科寒武纪科技股份有限公司 用于执行人工神经网络正向运算的装置和方法
US10552119B2 (en) 2016-04-29 2020-02-04 Intel Corporation Dynamic management of numerical representation in a distributed matrix processor architecture
US11055063B2 (en) 2016-05-02 2021-07-06 Marvell Asia Pte, Ltd. Systems and methods for deep learning processor
US10187568B1 (en) 2016-05-02 2019-01-22 Bao Tran Video smart phone
CN105978611B (zh) 2016-05-12 2019-09-17 京信通信系统（中国）有限公司 一种频域信号压缩方法及装置
US9918006B2 (en) * 2016-05-20 2018-03-13 International Business Machines Corporation Device, system and method for cognitive image capture
AU2016203619A1 (en) 2016-05-31 2017-12-14 Canon Kabushiki Kaisha Layer-based operations scheduling to optimise memory for CNN applications
EP3252949B1 (en) 2016-06-01 2020-03-18 Intel IP Corporation Methods and devices for predistortion of signals
US20170357910A1 (en) 2016-06-10 2017-12-14 Apple Inc. System for iteratively training an artificial intelligence using cloud-based metrics
CN107545889B (zh) 2016-06-23 2020-10-23 华为终端有限公司 适用于模式识别的模型的优化方法、装置及终端设备
CN105979035B (zh) 2016-06-28 2019-08-27 Oppo广东移动通信有限公司 一种增强现实ar图像处理方法、装置及智能终端
CN106156310A (zh) 2016-06-30 2016-11-23 努比亚技术有限公司 一种图片处理装置和方法
US10372588B2 (en) 2016-07-08 2019-08-06 International Business Machines Corporation Providing debug information on production containers using debug containers
DE102016214786A1 (de) 2016-08-09 2018-02-15 Fujitsu Limited Anwendungsprofiling-Jobmanagement-System, -Programm und -Verfahren
CN107688855B (zh) 2016-08-12 2021-04-13 赛灵思公司 针对于复杂神经网络的分层量化方法与装置
US20180046903A1 (en) 2016-08-12 2018-02-15 DeePhi Technology Co., Ltd. Deep processing unit (dpu) for implementing an artificial neural network (ann)
CN106354568A (zh) 2016-08-23 2017-01-25 京信通信技术(广州)有限公司 一种不同进程间的通信方法及通信装置
CN107797913A (zh) 2016-09-07 2018-03-13 大陆汽车电子（连云港）有限公司 一种实时系统的软件分析系统与方法
US11907760B2 (en) 2016-09-23 2024-02-20 Apple Inc. Systems and methods of memory allocation for neural networks
CN106650922B (zh) 2016-09-29 2019-05-03 清华大学 硬件神经网络转换方法、计算装置、软硬件协作系统
US20180096243A1 (en) 2016-09-30 2018-04-05 General Electric Company Deep learning for data driven feature representation and anomaly detection
WO2018071546A1 (en) 2016-10-11 2018-04-19 The Research Foundation For The State University Of New York System, method, and accelerator to process convolutional neural network layers
CN106485316B (zh) 2016-10-31 2019-04-02 北京百度网讯科技有限公司 神经网络模型压缩方法以及装置
CN106502626A (zh) 2016-11-03 2017-03-15 北京百度网讯科技有限公司 数据处理方法和装置
US10216479B2 (en) 2016-12-06 2019-02-26 Arm Limited Apparatus and method for performing arithmetic operations to accumulate floating-point numbers
US10372496B2 (en) * 2017-01-06 2019-08-06 International Business Machines Corporation Optimizing timeouts and polling intervals
US10997492B2 (en) 2017-01-20 2021-05-04 Nvidia Corporation Automated methods for conversions to a lower precision data format
CN106951587A (zh) 2017-02-15 2017-07-14 芯启源（南京）半导体科技有限公司 Fpga调试系统及方法
US11032350B2 (en) * 2017-03-15 2021-06-08 Commvault Systems, Inc. Remote commands framework to control clients
CN106951962B (zh) 2017-03-22 2020-09-01 南京地平线机器人技术有限公司 用于神经网络的复合运算单元、方法和电子设备
US10402932B2 (en) 2017-04-17 2019-09-03 Intel Corporation Power-based and target-based graphics quality adjustment
US10332302B2 (en) 2017-04-17 2019-06-25 Intel Corporation Scatter gather engine
CN107025629B (zh) 2017-04-27 2021-03-26 维沃移动通信有限公司 一种图像处理方法及移动终端
US11842280B2 (en) 2017-05-05 2023-12-12 Nvidia Corporation Loss-scaling for deep neural network training with reduced precision
US10019668B1 (en) 2017-05-19 2018-07-10 Google Llc Scheduling neural network processing
EP3410245A1 (en) * 2017-06-02 2018-12-05 OMRON Corporation Process analysis apparatus, process analysis method, and process analysis program
US11144828B2 (en) 2017-06-09 2021-10-12 Htc Corporation Training task optimization system, training task optimization method and non-transitory computer readable medium for operating the same
US10944902B2 (en) 2017-06-20 2021-03-09 Adobe Inc. Digital image generation using capture support data
US20200097799A1 (en) 2017-06-30 2020-03-26 Intel Corporation Heterogeneous multiplier
CN107451654B (zh) 2017-07-05 2021-05-18 深圳市自行科技有限公司 卷积神经网络的加速运算方法、服务器及存储介质
US10427306B1 (en) 2017-07-06 2019-10-01 X Development Llc Multimodal object identification
CN109284130B (zh) 2017-07-20 2021-03-23 上海寒武纪信息科技有限公司 神经网络运算装置及方法
CN107451658B (zh) 2017-07-24 2020-12-15 杭州菲数科技有限公司 浮点运算定点化方法及系统
CN107688849B (zh) 2017-07-28 2021-04-13 赛灵思电子科技(北京)有限公司 一种动态策略定点化训练方法及装置
US11481218B2 (en) 2017-08-02 2022-10-25 Intel Corporation System and method enabling one-hot neural networks on a machine learning compute platform
EP3652681A4 (en) 2017-08-08 2020-07-29 Samsung Electronics Co., Ltd. PROCESS AND APPARATUS FOR DETERMINING MEMORY REQUIREMENTS IN A NETWORK
US20190050710A1 (en) 2017-08-14 2019-02-14 Midea Group Co., Ltd. Adaptive bit-width reduction for neural networks
CN107610042B (zh) 2017-08-23 2019-06-07 维沃移动通信有限公司 一种图像美化方法及移动终端
CN107644254A (zh) 2017-09-09 2018-01-30 复旦大学 一种卷积神经网络权重参数量化训练方法及系统
US10224954B1 (en) 2017-09-29 2019-03-05 Intel Corporation Floating point to fixed point conversion
US11437032B2 (en) * 2017-09-29 2022-09-06 Shanghai Cambricon Information Technology Co., Ltd Image processing apparatus and method
US10223114B1 (en) 2017-09-29 2019-03-05 Intel Corporation Fixed point to floating point conversion
JP6810283B2 (ja) * 2017-09-29 2021-01-06 シャンハイ カンブリコン インフォメーション テクノロジー カンパニー リミテッドＳｈａｎｇｈａｉ Ｃａｍｂｒｉｃｏｎ Ｉｎｆｏｒｍａｔｉｏｎ Ｔｅｃｈｎｏｌｏｇｙ Ｃｏ．，Ｌｔｄ． 画像処理装置及び方法
US11450319B2 (en) * 2017-09-29 2022-09-20 Cambricon (Xi'an) Semiconductor Co., Ltd. Image processing apparatus and method
JP6540770B2 (ja) 2017-10-17 2019-07-10 富士通株式会社 演算処理回路、演算処理回路を含む演算処理装置、演算処理装置を含む情報処理装置、および方法
US10410121B2 (en) 2017-10-25 2019-09-10 SparkCognition, Inc. Adjusting automated neural network generation based on evaluation of candidate neural networks
US20210061028A1 (en) 2017-10-26 2021-03-04 Applied Mechatronic Products Apparatus and method for vehicular monitoring, analysis, and control
CN107832845A (zh) 2017-10-30 2018-03-23 上海寒武纪信息科技有限公司 一种信息处理方法及相关产品
US10783634B2 (en) 2017-11-22 2020-09-22 General Electric Company Systems and methods to deliver point of care alerts for radiological findings
US10803379B2 (en) 2017-12-12 2020-10-13 Amazon Technologies, Inc. Multi-memory on-chip computational network
CN108053028B (zh) 2017-12-21 2021-09-14 深圳励飞科技有限公司 数据定点化处理方法、装置、电子设备及计算机存储介质
US11636327B2 (en) 2017-12-29 2023-04-25 Intel Corporation Machine learning sparse computation mechanism for arbitrary neural networks, arithmetic compute microarchitecture, and sparsity for training mechanism
US11373088B2 (en) 2017-12-30 2022-06-28 Intel Corporation Machine learning accelerator mechanism
KR102595790B1 (ko) * 2018-01-26 2023-10-30 삼성전자주식회사 전자 장치 및 그의 제어방법
CN108388446A (zh) 2018-02-05 2018-08-10 上海寒武纪信息科技有限公司 运算模块以及方法
US20190251429A1 (en) 2018-02-12 2019-08-15 Kneron, Inc. Convolution operation device and method of scaling convolution input for convolution neural network
US20200159532A1 (en) 2018-02-13 2020-05-21 Shanghai Cambricon Information Technology Co., Ltd Computing device and method
US11106598B2 (en) 2018-02-13 2021-08-31 Shanghai Cambricon Information Technology Co., Ltd. Computing device and method
US11630666B2 (en) 2018-02-13 2023-04-18 Shanghai Cambricon Information Technology Co., Ltd Computing device and method
CN110162162B (zh) 2018-02-14 2023-08-18 上海寒武纪信息科技有限公司 处理器的控制装置、方法及设备
JP7056225B2 (ja) 2018-02-26 2022-04-19 富士通株式会社 演算処理装置、情報処理装置、情報処理方法、およびプログラム
US10628275B2 (en) 2018-03-07 2020-04-21 Nxp B.V. Runtime software-based self-test with mutual inter-core checking
US11475306B2 (en) 2018-03-22 2022-10-18 Amazon Technologies, Inc. Processing for multiple input data sets
CN108510067B (zh) 2018-04-11 2021-11-09 西安电子科技大学 基于工程化实现的卷积神经网络量化方法
US11562213B2 (en) 2018-04-17 2023-01-24 Intel Corporation Methods and arrangements to manage memory in cascaded neural networks
US10691413B2 (en) 2018-05-04 2020-06-23 Microsoft Technology Licensing, Llc Block floating point computations using reduced bit-width vectors
EP3624020A4 (en) 2018-05-18 2021-05-05 Shanghai Cambricon Information Technology Co., Ltd CALCULATION PROCEDURES AND RELATED PRODUCTS
CN108717570A (zh) 2018-05-23 2018-10-30 电子科技大学 一种脉冲神经网络参数量化方法
CN110554500B (zh) 2018-05-31 2022-09-16 中强光电股份有限公司 头戴式显示装置
US10360304B1 (en) 2018-06-04 2019-07-23 Imageous, Inc. Natural language processing interface-enabled building conditions control system
CN109062540B (zh) 2018-06-06 2022-11-25 北京理工大学 一种基于cordic算法的可重构浮点运算装置
CN109063820A (zh) 2018-06-07 2018-12-21 中国科学技术大学 利用时频联合长时循环神经网络的数据处理方法
US20190044860A1 (en) * 2018-06-18 2019-02-07 Intel Corporation Technologies for providing adaptive polling of packet queues
CN110728364A (zh) 2018-07-17 2020-01-24 上海寒武纪信息科技有限公司 一种运算装置和运算方法
KR102470893B1 (ko) 2018-06-27 2022-11-25 상하이 캠브리콘 인포메이션 테크놀로지 컴퍼니 리미티드 온 칩 코드의 브레이크 포인트에 의한 디버그 방법, 온 칩 프로세서 및 브레이크 포인트에 의한 칩 디버그 시스템
EP3640810A4 (en) 2018-08-28 2021-05-05 Cambricon Technologies Corporation Limited DATA PRE-PROCESSING PROCESS AND APPARATUS, COMPUTER DEVICE AND STORAGE MEDIA
US11703939B2 (en) 2018-09-28 2023-07-18 Shanghai Cambricon Information Technology Co., Ltd Signal processing device and related products
CN109528197B (zh) * 2018-11-20 2022-07-08 中国科学院脑科学与智能技术卓越创新中心 基于脑功能图谱进行精神疾病的个体化预测方法和系统
US11436825B2 (en) * 2018-12-14 2022-09-06 Samsung Electronics Co., Ltd. Method and apparatus for determining target object in image based on interactive input
CN109685202B (zh) 2018-12-17 2023-03-21 腾讯科技（深圳）有限公司 数据处理方法及装置、存储介质和电子装置
CN111383638A (zh) * 2018-12-28 2020-07-07 上海寒武纪信息科技有限公司 信号处理装置、信号处理方法及相关产品
CN109902745A (zh) 2019-03-01 2019-06-18 成都康乔电子有限责任公司 一种基于cnn的低精度训练与8位整型量化推理方法
CN109993296B (zh) 2019-04-01 2020-12-29 安徽寒武纪信息科技有限公司 量化实现方法及相关产品
CN110059733A (zh) 2019-04-01 2019-07-26 苏州科达科技股份有限公司 卷积神经网络的优化及快速目标检测方法、装置
CN111832737B (zh) 2019-04-18 2024-01-09 中科寒武纪科技股份有限公司 一种数据处理方法及相关产品
US20200334522A1 (en) 2019-04-18 2020-10-22 Cambricon Technologies Corporation Limited Data processing method and related products
US11676028B2 (en) 2019-06-12 2023-06-13 Shanghai Cambricon Information Technology Co., Ltd Neural network quantization parameter determination method and related products
US11488226B2 (en) * 2019-06-24 2022-11-01 Tata Consultancy Limited Services Method and system for designing formulated products
US20210264270A1 (en) 2019-08-23 2021-08-26 Anhui Cambricon Information Technology Co., Ltd. Data processing method, device, computer equipment and storage medium
WO2021036908A1 (zh) 2019-08-23 2021-03-04 安徽寒武纪信息科技有限公司 数据处理方法、装置、计算机设备和存储介质
US20210374510A1 (en) 2019-08-23 2021-12-02 Anhui Cambricon Information Technology Co., Ltd. Data processing method, device, computer equipment and storage medium
WO2021036905A1 (zh) 2019-08-27 2021-03-04 安徽寒武纪信息科技有限公司 数据处理方法、装置、计算机设备和存储介质
US11720845B2 (en) * 2019-08-30 2023-08-08 Amplo Global Inc. Data driven systems and methods for optimization of a target business
CN110780845B (zh) 2019-10-17 2021-11-30 浙江大学 一种用于量化卷积神经网络的可配置近似乘法器及其实现方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105893419A (zh) * 2015-11-30 2016-08-24 乐视致新电子科技（天津）有限公司 一种多媒体照片生成方法、装置、设备及手机
CN106997236A (zh) * 2016-01-25 2017-08-01 亮风台（上海）信息科技有限公司 基于多模态输入进行交互的方法和设备
CN106557165A (zh) * 2016-11-14 2017-04-05 北京智能管家科技有限公司 智能设备的动作模拟交互方法和装置及智能设备
US20180190257A1 (en) * 2016-12-29 2018-07-05 Shadecraft, Inc. Intelligent Umbrellas and/or Robotic Shading Systems Including Noise Cancellation or Reduction
CN108389098A (zh) * 2017-02-03 2018-08-10 北京京东尚科信息技术有限公司 语音购物方法以及系统
CN106910503A (zh) * 2017-04-26 2017-06-30 海信集团有限公司 用于智能终端显示用户操控指令的方法、装置和智能终端
CN108509619A (zh) * 2018-04-04 2018-09-07 科大讯飞股份有限公司 一种语音交互方法及设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3859488A4 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111680527A (zh) * 2020-06-09 2020-09-18 语联网（武汉）信息技术有限公司 基于专属机翻引擎训练的人机共译系统与方法
CN111680527B (zh) * 2020-06-09 2023-09-19 语联网（武汉）信息技术有限公司 基于专属机翻引擎训练的人机共译系统与方法
US20230386470A1 (en) * 2021-01-06 2023-11-30 Boe Technology Group Co., Ltd. Speech instruction recognition method, electronic device, and non-transient computer readable storage medium

Also Published As

Publication number Publication date
EP3859488A4 (en) 2022-06-29
US11703939B2 (en) 2023-07-18
US20210341989A1 (en) 2021-11-04
EP3859488A1 (en) 2021-08-04

Similar Documents

Publication Publication Date Title
WO2020062392A1 (zh) 信号处理装置、信号处理方法及相关产品
US11544059B2 (en) Signal processing device, signal processing method and related products
JP6893968B2 (ja) 画像処理装置及び方法
US11450319B2 (en) Image processing apparatus and method
US11437032B2 (en) Image processing apparatus and method
US10521264B2 (en) Data processing architecture for improved data flow
CN110968235B (zh) 信号处理装置及相关产品
Wang et al. Real-time block-based embedded CNN for gesture classification on an FPGA
CN109584864A (zh) 图像处理装置和方法
Yuan et al. CTIF-Net: A CNN-Transformer Iterative Fusion Network for Salient Object Detection
CN110969246A (zh) 信号处理装置及相关产品
CN109785843A (zh) 图像处理装置和方法
CN104460991A (zh) 一种基于数字家庭设备的手势互动控制系统
CN110968285A (zh) 信号处理装置及相关产品
CN111767710A (zh) 印尼语的情感分类方法、装置、设备及介质
CN109584862B (zh) 图像处理装置和方法
WO2023226783A1 (zh) 一种数据处理方法及装置
Su Deep Neural Networks for 3D Processing and High-Dimensional Filtering
Pavitra et al. Review on Smart Music Player and Algorithm for Recommending Music Based on Facial Micro-Expressions with Multi Cultural Facial Expression Analysis
CN117765341A (zh) 一种数据处理方法及相关装置
CN117350913A (zh) 一种数据处理方法及其装置
CN117251592A (zh) 一种数据处理方法及其装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18920196

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2018920196

Country of ref document: EP

Effective date: 20210428