CN110348291A - A kind of scene recognition method, a kind of scene Recognition device and a kind of electronic equipment - Google Patents

Abstract

This application discloses a kind of scene recognition method, a kind of scene Recognition device and a kind of electronic equipment, are related to technical field of information processing, can solve the low problem of existing scene recognition method scene Recognition accuracy.The application is by combining scene recognizer, and temporal information, location information, Weather information and temperature information of image etc. carries out image scene identification, it can be to avoid problem be misidentified by caused by algorithm progress scene Recognition merely, to improve the accuracy of image recognition.

Description

A kind of scene recognition method, a kind of scene Recognition device and a kind of electronic equipment

Technical field

The invention relates to technical field of information processing more particularly to a kind of scene recognition method, a kind of scene to know Other device and a kind of electronic equipment.

Background technique

At present artificial intelligence (Artificial Intelligence, AI) using more and more extensive.One of which uses Scene is that the neural network of analytic learning is carried out by simulation human brain, carries out the scene that wisdom knows object.For example, according to plant picture Identify the plant variety or classification；Or identify the scene (such as snow scenes) of the preview screen shown in preview interface when taking pictures.

Existing neural network when carrying out scene Recognition, be based on training after neural network carry out intelligent recognition. For example, neural network algorithm is after a large amount of flowers picture training, can be according to training when preview practical flowers after Neural network algorithm classify, and then identify result.But this method only relies on algorithm progress, therefore recognition result can It is lower by property.For example, the scene to be descended slowly and lightly due to oriental cherry and the scene to snow are very close, neural network can descend slowly and lightly oriental cherry scene It is mistakenly identified as the scene that snows.

Summary of the invention

The embodiment of the present application provides a kind of scene recognition method, a kind of scene Recognition device and a kind of electronic equipment, can be with Improve the accuracy of image recognition.

In order to achieve the above objectives, the embodiment of the present application adopts the following technical scheme that

In a first aspect, a kind of scene recognition method is provided, this method comprises: the first image of identification, determines first image Scene Recognition as a result, in the scene Recognition result include at least one scene type；Obtain bat when acquiring first image Information is taken the photograph, which includes at least: one or more of temporal information, location information, Weather information and temperature information； According to photographing information, determine that the label of first image, the label of first image are used to indicate this from scene Recognition result The scene type of first image.

The technical solution that above-mentioned first aspect provides, by combine the temporal information of image, location information, Weather information and Temperature information etc. carries out image scene identification, can misidentify problem caused by algorithm progress scene Recognition to avoid relying on merely, To improve the accuracy of image recognition.

In one possible implementation, at least one above-mentioned scene type, includes at least: the image background in image One or more of the corresponding season information of information, image, the corresponding Weather information of image and reference object information of image. The application classifies to the shooting background of image, season, weather or reference object etc. by the photographing information in conjunction with image, To improve the accuracy of image recognition.

In one possible implementation, at least one above-mentioned scene type is according to the first image and each scene type The descending sequence of matching degree.Processing in this way, so as to combine image photographing information, by with the image Scene type of the highest scene type of photographing information matching degree as the first image.

In one possible implementation, according to photographing information, the mark of the first image is determined from scene Recognition result Label, comprising: descending according to the matching degree and at least one scene type of photographing information and at least one scene type Sequence, determine the label of the first image.The is determined according to the matching degree of the photographing information of image and each scene type The scene type of one image, can be to avoid problem be misidentified by caused by algorithm progress scene Recognition merely, to improve figure As the accuracy of identification.

In one possible implementation, the scene recognition method of the application can be applied to include Processing with Neural Network The electronic equipment of unit NPU chip；The first image of the identification, determines the scene Recognition result of the first image, comprising: pass through NPU chip identifies the first image, determines the scene Recognition result of the first image.The scene recognition method of the application can pass through NPU chip is realized.

In one possible implementation, Cambrian Cambricon instruction set is integrated in NPU chip；The NPU chip Accelerate the process of the scene Recognition result of determining first image using Cambrian Cambricon instruction set.By using The speed of scene Recognition can be improved in Cambricon instruction set, improves user experience.

In one possible implementation, the first image is the preview image of the camera acquisition of electronic equipment.This Shen Scene recognition method please can be what the preview image acquired for camera carried out.

In one possible implementation, the first image is stored picture；Alternatively, the first image is set from other The standby picture obtained.The scene recognition method of the application is also possible to for the progress of existing picture, including electronics is used to set Standby shooting, and obtained from third party.

In one possible implementation, according to photographing information, the first image is determined from scene Recognition result After label, this method further include: the acquisition parameters of camera are adjusted, so that the tag match of acquisition parameters and the first image. By improving the accuracy of preview image scene Recognition result, suitable acquisition parameters shoot the preview graph so as to adjust Picture obtains better shooting effect, improves user experience.

In one possible implementation, convolutional neural networks, this method are integrated in above-mentioned NPU chip further include: By the tag update of the first image and the first image into the training set of convolutional neural networks；It is instructed again according to updated training set Practice convolutional neural networks.It, can be with by using each image and the corresponding label re -training convolutional neural networks of the image The algorithm of convolutional neural networks is constantly improve, the accuracy that convolutional neural networks carry out scene Recognition is improved.

Second aspect provides a kind of scene Recognition device, which includes: scene Recognition unit, for knowing Other first image determines the scene Recognition of first image as a result, including at least one scene type in the scene Recognition result； Information acquisition unit acquires photographing information when first image for obtaining, which includes at least: temporal information, One or more of location information, Weather information and temperature information；The scene Recognition unit is also used to, according to photographing information, Determine that the label of the first image, the label of first image are used to indicate the scene class of the first image from scene Recognition result Not.

The technical solution that above-mentioned second aspect provides, by combine the temporal information of image, location information, Weather information and Temperature information etc. carries out image scene identification, can misidentify problem caused by algorithm progress scene Recognition to avoid relying on merely, To improve the accuracy of image recognition.

In one possible implementation, at least one above-mentioned scene type, includes at least: the image background in image One or more of the corresponding season information of information, image, the corresponding Weather information of image and reference object information of image. The application classifies to the shooting background of image, season, weather or reference object etc. by the photographing information in conjunction with image, To improve the accuracy of image recognition.

In one possible implementation, at least one above-mentioned scene type is according to the first image and each scene type The descending sequence of matching degree.Processing in this way, so as to combine image photographing information, by with the image Scene type of the highest scene type of photographing information matching degree as the first image.

In one possible implementation, scene Recognition unit is determined from scene Recognition result according to photographing information The first image label, comprising: the scene Recognition unit according to the matching degree of photographing information and at least one scene type, And the sequence that at least one scene type is descending, determine the label of the first image.According to the photographing information of image and often The matching degree of one scene type determines the scene type of the first image, can be to avoid merely by algorithm progress scene Recognition Caused misrecognition problem, to improve the accuracy of image recognition.

In one possible implementation, scene Recognition unit includes neural-network processing unit NPU chip；Above-mentioned field Scape recognition unit identifies the first image, determines the scene Recognition result of the first image, comprising: the scene Recognition unit passes through NPU Chip identifies the first image, determines the scene Recognition result of the first image.The scene recognition method of the application can pass through NPU core Piece is realized.

In one possible implementation, Cambrian Cambricon instruction set is integrated in NPU chip；The NPU chip Accelerate the process of the scene Recognition result of determining first image using Cambrian Cambricon instruction set.By using The speed of scene Recognition can be improved in Cambricon instruction set, improves user experience.

In one possible implementation, scene Recognition device further include: camera, the first image are camera acquisitions Preview image.The scene recognition method of the application can be the preview image progress for the acquisition of scene Recognition device camera 's.

In one possible implementation, the first image is stored picture；Alternatively, the first image is set from other The standby picture obtained.The scene recognition method of the application is also possible to for the progress of existing picture, including electronics is used to set Standby shooting, and obtained from third party.

In one possible implementation, the device further include: parameter adjustment unit, in scene recognition unit root The acquisition parameters of camera are adjusted after the label for determining the first image in scene Recognition result according to photographing information, so that clapping Take the photograph the tag match of parameter Yu the first image.By improving the accuracy of preview image scene Recognition result, so as to adjust Suitable acquisition parameters shoot the preview image, obtain better shooting effect, improve user experience.

In one possible implementation, convolutional neural networks, above-mentioned scene Recognition list are integrated in above-mentioned NPU chip Member is also used to: by the tag update of the first image and the first image into the training set of convolutional neural networks；According to updated instruction Practice collection re -training convolutional neural networks.By using each image and the corresponding label re -training convolutional Neural of the image Network can constantly improve the algorithm of convolutional neural networks, improve the accuracy that convolutional neural networks carry out scene Recognition.

The third aspect provides a kind of user equipment (UE), which includes: scene Recognition device, for realizing such as first aspect The scene recognition method in any possible implementation.

Fourth aspect provides a kind of user equipment (UE), which includes: memory, for storing computer program code, institute Stating computer program code includes instruction；Radio circuit, for carrying out sending and receiving for wireless signal；Processor, for holding Row described instruction is realized such as the scene recognition method in any possible implementation of first aspect.

5th aspect, provides a kind of computer readable storage medium, calculating is stored on the computer readable storage medium Machine executes instruction, and is realized when which is executed by processor as in any possible implementation of first aspect Scene recognition method.

6th aspect, provides a kind of chip system, which includes processor, memory, is stored in memory Instruction；When described instruction is executed by the processor, realize that the scene in any possible implementation of first aspect such as is known Other method.The chip system can be made of chip, also may include chip and other discrete devices.

Detailed description of the invention

Fig. 1 is a kind of course of work schematic diagram of convolutional neural networks provided by the embodiments of the present application；

Fig. 2 is a kind of pond method schematic diagram provided by the embodiments of the present application；

Fig. 3 is a kind of mobile phone hardware structural schematic diagram provided by the embodiments of the present application；

Fig. 4 is a kind of scene recognition method flow chart one provided by the embodiments of the present application；

Fig. 5 is a kind of scene recognition method flowchart 2 provided by the embodiments of the present application；

Fig. 6 is a kind of accelerator architecture based on Cambricon instruction set provided by the embodiments of the present application；

Fig. 7 is a kind of scene recognition method flow chart 3 provided by the embodiments of the present application；

Fig. 8 is a kind of scene recognition method flow chart four provided by the embodiments of the present application；

Fig. 9 is a kind of structural schematic diagram of mobile phone provided by the embodiments of the present application.

Specific embodiment

The embodiment of the present application provides a kind of scene recognition method, a kind of scene Recognition device and a kind of electronic equipment.Specifically , for example, this method can be used for by convolutional Neural algorithm to images to be recognized carry out scene Recognition during.

Wherein, images to be recognized can refer to the picture shot, the preview of camera, from the picture obtained elsewhere Or a certain frame image in video etc..The embodiment of the present application to the source of images to be recognized, format and acquisition modes etc. no It limits.Scene Recognition result in the embodiment of the present application can identify image background information (such as night scene, snowfield, sandy beach), can To identify the corresponding season information of image (such as autumn), the corresponding Weather information of image (such as rainy day, cloudy day) can also be identified, also The reference object information of image can be identified (such as oriental cherry descends slowly and lightly, baby, snows).Above several scene Recognition result only conducts Several examples, the embodiment of the present application are not construed as limiting the concrete scene classification in specific scene Recognition result.

Following example is please referred to, to be that scene recognition method in the embodiment of the present application is several possible answer several examples below Use example.

Example 1: user is taken pictures using the camera of user equipment (User Equipment, UE), in preview interface, UE Scene Recognition is carried out to preview image.Acquisition parameters based on scene Recognition result adjustment camera.It is clapped in this way, being clicked in user According to button, when shooting the preview image, can be shot with scene style, the color etc. the most based on acquisition parameters adjusted The picture matched, shooting effect is more preferable, and user experience is more preferable.

Wherein, preview interface refers to that UE enables the interface that camera preview currently wants shooting picture.UE after starting camera, The current preview interface of camera can be shown on the display screen of UE, so that user determines whether current picture is user's picture to be shot Face.

Example 2: existing picture is uploaded to a certain website by user, and the scene type of the picture is identified by the website.Tool Body, the scene type of the picture is identified by the Website server of the website.Or user should by the APP identification installed in UE The scene type of picture.For example, user wishes to know the department (such as rosaceae) and title of the plant in the picture of its shooting (such as Chinese rose).The picture can be uploaded to a certain website by user, identify plant in the picture by the Website server of the website Department and title.

Example 3: user wishes to find the dress ornament that first is worn in a certain shopping APP.User can upload first and wear The photo of the dress ornament is to shopping APP.The identification of the dress ornament is completed by the application server of the APP, and from shopping APP Same money dress ornament is matched, user is recommended.

Example 4: user wishes that the picture 1 shot is rendered into the picture 2 for more meeting its scene.For example, user's self-timer Photo of one station in snow scenes, it is desirable to render more dreamlike snow scenes in the background.User can be by picture at Class APP is managed, identifies the scene of the picture 1.And by the APP after obtaining scene Recognition result, according to the scene Recognition result Picture 1 is rendered, better snow scenes effect is obtained.

Know it should be noted that above-mentioned example 1- example 4 only introduces the scene in the embodiment of the present application as several examples The other possible several applications of method.Scene recognition method in the embodiment of the present application can also be applied to other possible situations In, the embodiment of the present application does not limit this.

It, can be in addition, electronic equipment in the embodiment of the present application can be smart phone, tablet computer, smart camera For other desktop types, on knee, hand-held type device, such as net book, personal digital assistant (Personal Digital Assistant, PDA), wearable device (such as smartwatch), AR (augmented reality)/VR (virtual reality) equipment etc., can also Think server category equipment (such as example 2 and example 3) or other equipment.The embodiment of the present application to the type of electronic equipment not It limits.

In one possible implementation, convolutional neural networks can integrate in electronic equipment.For example, electronic equipment Convolutional neural networks be integrated in neural-network processing unit (neural-network processing unit, NPU) chip In, the scene recognition method of the embodiment of the present application is completed by NPU.Alternatively, the convolutional neural networks of electronic equipment integrate it is on the scene In scape identification device, the scene recognition method of the embodiment of the present application is completed by scene Recognition device.

Wherein, convolutional neural networks are a kind of feedforward neural networks, and artificial neuron can respond surrounding cells, Ke Yijin Row large size image procossing.Convolutional neural networks include full-mesh layer (the corresponding classical mind on one or more convolutional layers and top Through network), while also including associated weights and pond layer (pooling layer).This structure enables convolutional neural networks Enough utilize the two-dimensional structure of input data.Compared with other deep learning structures, convolutional neural networks are in image and speech recognition Aspect can provide better result.This model also can be used back-propagation algorithm and be trained.Compare other depth, Feedforward neural network, the parameter that convolutional neural networks need to consider is less, it is a kind of deep learning structure for having much attraction.

As shown in Figure 1, being a kind of course of work schematic diagram of convolutional neural networks.As shown in Figure 1,120 logarithm of convolutional layer It is obtained according to input layer (Input layer) 110 and by pretreatment (for example, pretreatment includes going mean value, normalization and principal component Analysis (principal component analysis, PCA)/albefaction (whitening)) image data carry out feature mention It takes.130 pairs of activation primitive layer are done Nonlinear Mapping the result that convolutional layer 120 exports.For example, activation primitive layer 130 is using sharp Function amendment linear unit (The Rectified Linear Unit, ReLU) is encouraged to be compressed to the result that convolutional layer 120 exports Some range fixed, it is controllable for being always maintained at the numberical range gone down in layer in this way.Wherein, the spy of ReLU Point is that convergence is fast, asks gradient simple.Then, layer 140 pairs of feature in pond sample, i.e., substitute one piece of region with a numerical value, Primarily to reducing the over-fitting degree of network training parameter and model.Finally, the spy that full articulamentum 150 extracts front Sign integrates.Since each node of full articulamentum 150 is connected with upper one layer of all nodes, have complete Connected characteristic, that is, being with the connection type of traditional neural network neuron.

Wherein, the method for pond layer 140 has Max pooling and average pooling.Wherein, Max pooling Refer to that the window for each 2 × 2 selects the value of respective element of the maximum number as output matrix.As shown in Fig. 2, input square Maximum number is 6 in first 2 × 2 window of battle array, then first element of output matrix is exactly 6, is so analogized.

The basic principle of scene recognition method in the embodiment of the present application is: scene Recognition equipment (including UE, server category Equipment or scene Recognition device) based on convolutional neural networks obtain images to be recognized scene Recognition as a result, binding time, position It sets, the information such as weather, humidity, temperature, the final label for determining images to be recognized.

Referring to FIG. 3, as shown in figure 3, being a kind of hardware structural diagram of mobile phone provided by the embodiments of the present application.Such as figure Shown in 3, mobile phone 300 may include processor 310, memory (including external memory interface 320 and internal storage 321), Universal serial bus (universal serial bus, USB) interface 330, charge management module 340, power management module 341, battery 342, antenna 1, antenna 2, mobile communication module 350, wireless communication module 360, audio-frequency module 370, loudspeaker 370A, microphone 370C, sensor module 380, key 390, indicator 392, camera 393, display screen 394 and user Mark module (subscriber identification module, SIM) card interface 395 etc..Wherein sensor module 380 can To include gyro sensor 380A, pressure sensor 380B, acceleration transducer 380C, temperature sensor 380D, touches and pass Sensor 380E, ambient light sensor 380F etc..

It is understood that the structure of signal of the embodiment of the present invention does not constitute the specific restriction to mobile phone 300.In this Shen Please in other embodiments, mobile phone 300 may include than illustrating more or fewer components, perhaps combine certain components or Split certain components or different component layouts.The component of diagram can be with hardware, and the combination of software or software and hardware is real It is existing.

Processor 310 may include one or more processing units, such as: processor 310 may include application processor (application processor, AP), modem processor, graphics processor (graphics processing Unit, GPU), image-signal processor (image signal processor, ISP), controller, Video Codec, number Signal processor (digital signal processor, DSP), baseband processor and/or neural network processor NPU core Piece etc..Wherein, different processing units can be independent device, also can integrate in one or more processors.

Controller can generate operating control signal according to instruction operation code and clock signal, complete instruction fetch and execution The control of instruction.

Memory can also be set in processor 310, for storing instruction and data.In some embodiments, processor Memory in 310 is cache memory.The memory can save the instruction that processor 310 is just used or is recycled Or data.If processor 310 needs to reuse the instruction or data, can be called directly from the memory.It avoids Repeated access, reduces the waiting time of processor 310, thus improves the efficiency of system.

In some embodiments, processor 310 may include one or more interfaces.Interface may include integrated circuit (inter-integrated circuit, I2C) interface, integrated circuit built-in audio (inter-integrated circuit Sound, I2S) interface, pulse code modulation (pulse code modulation, PCM) interface, universal asynchronous receiving-transmitting transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), universal input export (general-purpose Input/output, GPIO) interface, Subscriber Identity Module (subscriber identity module, SIM) interface, and/or Universal serial bus (universal serial bus, USB) interface etc..

Charge management module 340 is used to receive charging input from charger.Wherein, charger can be wireless charger, It is also possible to wired charger.In the embodiment of some wired chargings, charge management module 340 can pass through USB interface 330 Receive the charging input of wired charger.In the embodiment of some wireless chargings, charge management module 340 can pass through mobile phone 300 Wireless charging coil receives wireless charging input.While charge management module 340 is that battery 342 charges, it can also lead to Crossing power management module 341 is power electronic equipment.

Power management module 341 is for connecting battery 342, charge management module 340 and processor 310.Power management mould Block 341 receives the input of battery 342 and/or charge management module 340, is processor 310, internal storage 321, display screen 394, the power supply such as camera 393 and wireless communication module 360.Power management module 341 can be also used for monitoring battery capacity, Circulating battery number, the parameters such as cell health state (electric leakage, impedance).In some other embodiment, power management module 341 Also it can be set in processor 310.In further embodiments, power management module 341 and charge management module 340 can also To be set in the same device.

The wireless communication function of mobile phone 300 can pass through antenna 1, antenna 2, mobile communication module 350, wireless communication module 360, modem processor and baseband processor etc. are realized.

Antenna 1 and antenna 2 electromagnetic wave signal for transmitting and receiving.Each antenna in mobile phone 300 can be used for covering list A or multiple communication bands.Different antennas can also be multiplexed, to improve the utilization rate of antenna.Such as: antenna 1 can be multiplexed For the diversity antenna of WLAN.In other embodiments, antenna can be used in combination with tuning switch.

Mobile communication module 350 can provide the solution of wireless communications such as including 2G/3G/4G/5G applied on mobile phone 300 Certainly scheme.Mobile communication module 350 may include at least one filter, switch, power amplifier, low-noise amplifier (low Noise amplifier, LNA) etc..Mobile communication module 350 can receive electromagnetic wave by antenna 1, and to received electromagnetic wave It is filtered, the processing such as amplification is sent to modem processor and is demodulated.Mobile communication module 350 can also be to through adjusting The modulated signal amplification of demodulation processor processed, switchs to electromagenetic wave radiation through antenna 1 and goes out.In some embodiments, mobile logical At least partly functional module of letter module 350 can be arranged in processor 310.In some embodiments, mobile communication mould At least partly functional module of block 350 can be arranged in the same device at least partly module of processor 310.

Modem processor may include modulator and demodulator.Wherein, modulator is used for low frequency base to be sent Band signal is modulated into high frequency signal.Demodulator is used to received electromagnetic wave signal being demodulated into low frequency baseband signal.Then solution Adjust device that the low frequency baseband signal that demodulation obtains is sent to baseband processor.Low frequency baseband signal is through baseband processor Afterwards, it is delivered to application processor.Application processor is defeated by audio frequency apparatus (being not limited to loudspeaker 370A, receiver 370B etc.) Voice signal out, or image or video are shown by display screen 394.In some embodiments, modem processor can be Independent device.In further embodiments, modem processor can be independently of processor 310, with mobile communication module 350 or other function module be arranged in the same device.

It includes WLAN (wireless local that wireless communication module 360, which can be provided and be applied on mobile phone 300, Area networks, WLAN) (such as Wireless Fidelity (wireless fidelity, Wi-Fi) network), bluetooth (bluetooth, BT), Global Navigation Satellite System (global navigation satellite system, GNSS), frequency modulation (frequency Modulation, FM), the short distance wireless communication technology (near field communication, NFC), infrared technique The solution of wireless communications such as (infrared, IR).Wireless communication module 360 can be integrated into a few communication process mould One or more devices of block.Wireless communication module 360 receives electromagnetic wave via antenna 2, by electromagnetic wave signal frequency modulation and filter Wave processing, by treated, signal is sent to processor 310.Wireless communication module 360 can also receive pending from processor 310 The signal sent carries out frequency modulation to it, and amplification switchs to electromagenetic wave radiation through antenna 2 and goes out.

In some embodiments, the antenna 1 of mobile phone 300 and mobile communication module 350 couple, antenna 2 and radio communication mold Block 360 couples, and allowing mobile phone 300, technology is communicated with network and other equipment by wireless communication.The wireless communication Technology may include global system for mobile communications (global system for mobile communications, GSM), lead to With grouping wireless service (general packet radio service, GPRS), CDMA accesses (code division Multiple access, CDMA), wideband code division multiple access (wideband code division multiple access, WCDMA), time division CDMA (time-division code division multiple access, TD-SCDMA), it is long Phase evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC, FM and/or IR technology etc..The GNSS can To include GPS (global positioning system, GPS), Global Navigation Satellite System (global Navigation satellite system, GLONASS), Beidou satellite navigation system (beidou navigation Satellite system, BDS), quasi- zenith satellite system (quasi-zenith satellite system, QZSS) and/or Satellite-based augmentation system (satellite based augmentation systems, SBAS).

Mobile phone 300 realizes display function by GPU, display screen 394 and application processor etc..GPU is image procossing Microprocessor connects display screen 394 and application processor.GPU is calculated for executing mathematics and geometry, is rendered for figure.Place Managing device 310 may include one or more GPU, execute program instructions to generate or change display information.Specific to the application reality It applies in example, after determining scene Recognition result, picture can be rendered into the effect for being suitble to the picture tag by mobile phone 300 by GPU Fruit.

Display screen 394 is for showing image, video etc..Display screen 394 includes display panel.Display panel can use liquid Crystal display screen (liquid crystal display, LCD), Organic Light Emitting Diode (organic light-emitting Diode, OLED), active matrix organic light-emitting diode or active-matrix organic light emitting diode (active-matrix Organic light emitting diode's, AMOLED), Flexible light-emitting diodes (flex light-emitting Diode, FLED), Miniled, MicroLed, Micro-oLed, light emitting diode with quantum dots (quantum dot light Emitting diodes, QLED) etc..In some embodiments, mobile phone 300 may include 1 or N number of display screen 394, and N is big In 1 positive integer.

Mobile phone 300 can pass through ISP, camera 393, Video Codec, GPU, display screen 394 and application processor Deng realization shooting function.

ISP is used to handle the data of the feedback of camera 393.For example, opening shutter when taking pictures, light is passed by camera lens It is delivered on camera photosensitive element, optical signal is converted to electric signal, and camera photosensitive element passes to the electric signal at ISP Reason, is converted into macroscopic image.ISP can also be to the noise of image, brightness, colour of skin progress algorithm optimization.ISP can be with Exposure to photographed scene, the parameter optimizations such as colour temperature.In some embodiments, ISP can be set in camera 393.

Camera 393 is for capturing still image or video.Object generates optical imagery by camera lens and projects photosensitive member Part.Photosensitive element can be charge-coupled device (charge coupled device, CCD) or complementary metal oxide is partly led Body (complementary metal-oxide-semiconductor, CMOS) phototransistor.Photosensitive element turns optical signal It changes electric signal into, electric signal is passed into ISP later and is converted into data image signal.Data image signal is output to DSP by ISP Working process.Data image signal is converted into the RGB of standard, the picture signal of the formats such as YUV by DSP.In some embodiments, Mobile phone 300 may include 1 or N number of camera 393, and N is the positive integer greater than 1.

Digital signal processor, in addition to can handle data image signal, can also handle it for handling digital signal His digital signal.For example, digital signal processor is used to carry out Fourier to frequency point energy when mobile phone 300 is when frequency point selects Transformation etc..

Video Codec is used for compression of digital video or decompression.Mobile phone 300 can support one or more videos Codec.In this way, mobile phone 300 can play or record the video of a variety of coded formats, and such as: dynamic image expert group (moving picture experts group, MPEG) 1, MPEG2, mpeg 3, MPEG4 etc..

NPU is neural-network processing unit (Neural-network Processing Unit), by using for reference biology mind Through network structure, such as transfer mode between human brain neuron is used for reference, to input information fast processing, can also constantly learnt by oneself It practises.The application such as intelligent cognition of mobile phone 300 may be implemented by NPU, such as: image recognition, recognition of face, scene Recognition, language Sound identification, text understanding etc..Specific in the embodiment of the present application, NPU can be understood as the unit for being integrated with convolutional neural networks, Or it can be understood as scene Recognition device.Or can be understood as scene Recognition device may include NPU, for to be identified Image carries out scene Recognition.

External memory interface 320 can be used for connecting external memory card, such as Micro SD card, realize extended mobile phone 300 storage capacity.External memory card is communicated by external memory interface 320 with processor 310, realizes that data store function Energy.Such as by music, the files such as video are stored in external memory card.

Internal storage 321 can be used for storing computer executable program code, and the executable program code includes Instruction.Internal storage 321 may include storing program area and storage data area.Wherein, storing program area can store operation system It unites, application program (such as sound-playing function, image player function etc.) needed at least one function etc..It storage data area can Data (such as audio data, phone directory etc.) created in 300 use process of memory mobile phone etc..In addition, internal storage 321 May include high-speed random access memory, can also include nonvolatile memory, a for example, at least disk memory, Flush memory device, generic flash memory (universal flash storage, UFS) etc..Processor 310 passes through operation storage In the instruction of internal storage 321, and/or it is stored in the instruction for the memory being set in processor, executes each of mobile phone 300 Kind functional application and data processing.

Mobile phone 300 can pass through audio-frequency module 370, loudspeaker 370A, receiver 370B, microphone 370C, earphone interface 370D and application processor etc. realize audio-frequency function.Such as music, recording etc..

Audio-frequency module 370 is used to for digitized audio message to be converted into analog audio signal output, is also used for analogue audio frequency Input is converted to digital audio and video signals.Audio-frequency module 370 can be also used for audio-frequency signal coding and decoding.

Loudspeaker 370A, also referred to as " loudspeaker ", for audio electrical signal to be converted to voice signal.Mobile phone 300 can pass through Loudspeaker 370A listens to music, or listens to hand-free call.

Receiver 370B, also referred to as " earpiece ", for audio electrical signal to be converted into voice signal.When mobile phone 300 answers electricity It, can be by the way that receiver 370B be answered voice close to human ear when words or voice messaging.

Microphone 370C, also referred to as " microphone ", " microphone ", for voice signal to be converted to electric signal.When making a phone call Or when sending voice messaging, voice signal can be input to microphone by mouth close to microphone 370C sounding by user 370C.At least one microphone 370C can be set in mobile phone 300.

Earphone interface 370D is for connecting wired earphone.Earphone interface 370D can be USB interface 330, be also possible to Opening mobile electronic device platform (open mobile terminal platform, OMTP) standard interface of 3.5mm, the U.S. Cellular telecommunication industrial association (cellular telecommunications industry association of the USA, CTIA) standard interface.

Gyro sensor 380A is determined for the athletic posture of mobile phone 300.In some embodiments, can pass through Gyro sensor 380A determines that mobile phone 300 surrounds the angular speed of three axis (that is, x, y and z-axis).Gyro sensor 380A can For shooting stabilization.Specific in embodiments herein, if mobile phone 300 passes through the collected hand of gyro sensor 380A The scene Recognition result of the data combination convolutional neural networks such as the angle that machine 300 is shaken determines that the scene of present preview image is to jump Umbrella, mobile phone 300 can go out the distance that lens module needs to compensate according to the angle calculation that mobile phone 300 is shaken, allow camera lens to pass through anti- Stabilization is realized in shake to balancing out motions mobile phone 300.Gyro sensor 380A can be also used for navigating, somatic sensation television game scene.

Pressure sensor 380B is for measuring pressure or pressure.For example, what mobile phone 300 was measured by pressure sensor 380B Atmospheric pressure value calculates height above sea level, determines current preview in conjunction with the scene Recognition result of cellphone GPS positioning and convolutional neural networks The scene of image is Yulong Xueshan, and the adjustable acquisition parameters of mobile phone 300 make it be more suitable for the shooting of current scene.

Acceleration transducer 380C can detect the size of (the generally three axis) acceleration in all directions of mobile phone 300.When Mobile phone 300 can detect that size and the direction of gravity when static.It can be also used for identification electronic equipment posture, be applied to horizontal/vertical screen Switching, the application such as pedometer.Illustratively, specific in embodiments herein, if mobile phone 300, which combines, passes through acceleration sensing The data such as the size of collected 300 gravity of mobile phone of device 380C and direction pass through the collected number pressure of pressure sensor 380B According to, in conjunction with the scene Recognition results of convolutional neural networks determine that the scene of present preview image is sea floor world, mobile phone 300 can be with Acquisition parameters are adjusted, it is made to be more suitable for underwater photograph technical.

Ambient light sensor 380F is for perceiving environmental light brightness.Illustratively, specific to embodiments herein, mobile phone 300 can be according to the collected environmental light brightness of ambient light sensor 380F, in conjunction with the scene Recognition result of convolutional neural networks The scene for determining current preview picture is night, and mobile phone 300 can be shot with light filling, and specific light filling amount can also regard environment Depending on the collected environmental light brightness of optical sensor 380F.

Temperature sensor 380D is for detecting temperature.Illustratively, specific to embodiments herein, mobile phone 300 can be with According to the collected temperature of temperature sensor 380D, the field of picture shooting is determined in conjunction with the scene Recognition result of convolutional neural networks Scape is that oriental cherry descends slowly and lightly rather than snows, which can be rendered into the atmosphere of winter snow scenes by mobile phone 300.

Touch sensor 380E, also referred to as " touch-control device ".Touch sensor 380E (also referred to as touch panel) can be set In display screen 394, touch screen is formed by touch sensor 380E and display screen 394, also referred to as " touch screen ".Touch sensor 380E It is applied to it or neighbouring touch operation for detecting.The touch operation that touch sensor can will test passes to application Processor, to determine touch event type.Visual output relevant to touch operation can be provided by display screen 394.Another In some embodiments, touch sensor 380E also be can be set in the surface of mobile phone 300, not with 394 location of display screen Together.Specific in the application, mobile phone 300 can detecte user in the pressing operation of the virtual shooting button of display screen 394, and In response to the operation, present preview image is shot.

Key 390 includes power button, volume key etc..Key 390 can be mechanical key.It is also possible to touch-key. Mobile phone 300 can receive key-press input, generate key signals input related with the user setting of mobile phone 300 and function control.

Motor 391 can produce vibration prompt.Motor 391 can be used for calling vibration prompt, can be used for touching vibration Dynamic feedback.For example, acting on the touch operation of different application (such as taking pictures, audio broadcasting etc.), different vibrations can be corresponded to Feedback effects.The touch operation of 394 different zones of display screen is acted on, motor 391 can also correspond to different vibrational feedback effects. Different application scenarios (such as: time alarm receives information, alarm clock, game etc.) different vibrational feedback effects can also be corresponded to Fruit.Touch vibrational feedback effect can also be supported customized.

Indicator 392 can be indicator light, can serve to indicate that charged state, electric quantity change can be used for instruction and disappear Breath, missed call, notice etc..

SIM card interface 395 is for connecting SIM card.SIM card can be by being inserted into SIM card interface 395, or from SIM card interface 395 extract, and realization is contacting and separating with mobile phone 300.Mobile phone 300 can support that 1 or N number of SIM card interface, N are greater than 1 Positive integer.SIM card interface 395 can support Nano SIM card, Micro SIM card, SIM card etc..The same SIM card interface 395 It can be inserted into multiple cards simultaneously.The type of multiple cards may be the same or different.SIM card interface 395 can also be compatible with Different types of SIM card.SIM card interface 395 can also be with compatible external storage card.Mobile phone 300 passes through SIM card and network interaction, Realize the functions such as call and data communication.In some embodiments, mobile phone 300 uses eSIM, it may be assumed that embedded SIM card.eSIM Card can cannot separate in mobile phone 300 with mobile phone 300.

Below in conjunction with the mobile phone in Fig. 3, scene recognition method provided by the embodiments of the present application is specifically introduced.Following embodiment In method can be realized in the mobile phone 300 with above-mentioned hardware configuration.

It should be understood that in the embodiment of the present application, some or all of mobile phone 300 can execute in the embodiment of the present application step Suddenly, these steps or operation are only examples, and the deformation of other operations or various operations can also be performed in the embodiment of the present application.This Outside, each step can be executed according to the different sequences that the embodiment of the present application is presented, and it is possible to not really want to execute sheet Apply for all operationss in embodiment.

As shown in figure 4, the scene recognition method in the embodiment of the present application can be realized by S401-S403:

S401, mobile phone 300 identify the first image, determine the scene Recognition result of the first image.

Wherein, the first image can be understood as images to be recognized above.First image can be user and pass through local The picture of camera shooting.It is installed in mobile phone for example, photo or user that user is directly shot by mobile phone camera pass through A certain application program (Application, APP) calling mobile phone camera shooting photo.First image is also possible to use The picture that family is got from other equipment.For example, the picture that snows that user is received by wechat from friend, user is from internet The oriental cherry of downloading descends slowly and lightly picture.Alternatively, first image can also be the image in other sources.For example, in the video of user's record A certain frame image.

In some embodiments, neural-network processing unit NPU chip has been can integrate in mobile phone 300.Convolutional Neural net Network can integrate in the NPU chip.

Mobile phone 300 identifies the first image, determines the scene Recognition of the first image as a result, may include: mobile phone 300 by first Image inputs convolutional neural networks, and the scene Recognition result of the first image is determined by convolutional neural networks.

Wherein, convolutional neural networks can train in advance before mobile phone 300 dispatches from the factory, and be solidificated in mobile phone 300.It can also be with Using mobile phone 300, captured photo or received, downloading picture are as training set within a preset period of time, to convolution Neural network carries out personalized training, so that accuracy of the convolutional neural networks when carrying out scene Recognition.For example, due to Family often shoots the photo of mountains and rivers plant, since mobile phone 300 constantly trains training set using the photo that user shoots, hand Machine 300 is higher for the scene Recognition result precision of mountains and rivers plant.

It wherein, may include at least one scene type in scene Recognition result.If scene Recognition result includes N number of scene Classification, N number of scene type can be according to the descending sequences of matching degree of each scene type and the first image.Wherein, N It is integer greater than 1, N.

Wherein, the matching degree of each scene type and the first image can refer to each scene in convolutional Neural net The successful match rate of corresponding feature and feature in the first image in the training set of network.Alternatively, N number of scene type can also foundation Other factors ranking, the embodiment of the present application are not construed as limiting specific ranking rule, method etc..

S402, mobile phone 300 obtain photographing information when the first image of acquisition.

Wherein, which is used to identify environmental information when acquiring the first image.The photographing information includes but unlimited In one or more of following information: temporal information, location information, Weather information and temperature information.

Several specific examples are exemplified below the acquisition of mobile phone 300 photographing information is specifically introduced:

Example (A): mobile phone 300 determines preview graph by convolutional neural networks and photographing information after starting camera The label of picture, and then select acquisition parameters corresponding with the label.

In this example, photographing information is that mobile phone 300 acquires.For example, mobile phone 300 can by with internet synchronize obtain Take current time information；Current location information is obtained by GPS；Pass through acquisition current weather information synchronous with internet；Pass through Humidity sensor obtains current humidity information；By temperature sensor 180D, or Current Temperatures information is obtained from network；It is logical It crosses gyro sensor 180A or acceleration transducer 180C determines current kinetic posture information；It is obtained by pressure sensor 180B Take current altitude information；Current environmental light brightness information etc. is obtained by ambient light sensor 180F.

Example (B): user wishes to know that the department (such as Calycanthaceae) of the plant in the picture of its shooting and title are (such as cured Plum).User can identify the department and title of plant in the picture by the APP installed in UE.

In this example, photographing information is the acquisition of mobile phone 300, and when inputting the picture to APP, while informing should The collected photographing information of APP mobile phone 300.Wherein, the ways and means that mobile phone 300 acquires photographing information can refer to but unlimited In ways and means cited hereinabove.

Example (C): user receives the opening and closing shadow of the user and friend by wechat from friend, and user wishes to pass through figure Piece handles APP and carries out background rendering to the group photo.In this example, photographing information is adopted by capture apparatus when shooting the group photo Collection, and recorded together with the group photo photographic intelligence.

It should be noted that above only as the possible acquisition modes of several illustration photographing informations and approach.Mobile phone 300 can also determine that photographing information, the embodiment of the present application are not construed as limiting this by other methods.

S403, mobile phone 300 determine the label of the first image according to photographing information from scene Recognition result.

Wherein, the label of the first image is used for the scene type of the first image.The scene type includes but is not limited to following One or more of: image background information, the corresponding season information of the first image in the first image, the first image are corresponding The reference object information of Weather information, the first image.

For example, for examples detailed above (A), convolutional neural networks of the mobile phone 300 after starting camera, in mobile phone 300 Scene Recognition is carried out to preview image, and is sorted according to the scene type in scene Recognition result, the bat obtained in conjunction with mobile phone 300 Take the photograph the label that information determines preview image.In another example the shopping class APP in mobile phone 300 can be called for examples detailed above (C) Convolutional neural networks in mobile phone 300 sort according to the scene type in scene Recognition result, determine the conjunction in conjunction with photographing information The label of shadow is snow scenes.

In one possible implementation, S403 can be realized by following procedure:

Mobile phone 300 determines the matching degree of each scene type in photographing information and scene Recognition result.Mobile phone 300 According to the matching degree sequence of each scene type in scene Recognition result, in conjunction with of photographing information and each scene type The label of the first image is determined with degree.

For example, in examples detailed above (A).The scene Recognition result of mobile phone 300 include two scene types, " Sea World " and " seabed ", and matching degree " Sea World " > " seabed " of two scene types and the first image.But photographing information is shown Current pressure is 5 × 10⁶Pascal (Pa), position are the Bohai Sea.Therefore, although scene type " Sea World " and the first image Matching degree is higher, but photographing information and the matching degree in " seabed " are greater than the matching journey of photographing information and " Sea World " Degree.Therefore, mobile phone 300 determines that the scene type is not " Sea World " in conjunction with photographing information, but " seabed ".I.e. this first not The label of image is " seabed ".

For example, in examples detailed above (B).The scene Recognition result of mobile phone 300 includes two scene types, " wintersweet " and " is met Spring flower ", and matching degree " winter jasmine " > " wintersweet " of two scene types and the first image.But photographing information is shown currently When the date be on December 25th, 2018, temperature be -5 DEG C.Photographing information and the matching degree of " winter jasmine " are 5%, photographing information Matching degree with " wintersweet " is 95%.Therefore, although scene type " winter jasmine " and the matching degree of the first image are higher, Being mobile phone 300 determines that the scene type is not " winter jasmine " in conjunction with photographing information, but " wintersweet ".That is the label of first image For " wintersweet ".

In another example in examples detailed above (C).The scene Recognition result of mobile phone 300 include two scene types, " snowing " and " oriental cherry descends slowly and lightly ", and the matching degree of two scene types and the first image " snows " > " oriental cherry descends slowly and lightly ".But photographing information Show that current location is in the park of Shanghai, the time is morning 10:00 on April 15th, 2019, and temperature is 23 DEG C, and weather is fine. Photographing information and the matching degree of " snowing " are 20%, and the matching degree of photographing information and " oriental cherry descends slowly and lightly " is 90%.Therefore, though Right scene type " snowing " and the matching degree of the first image are higher, but mobile phone 300 determines the scene class in conjunction with photographing information It is not " snowing ", but " oriental cherry descends slowly and lightly ".I.e. the label of first image is " oriental cherry descends slowly and lightly ".

It, can be to avoid merely by recognition result caused by algorithm based on scene recognition method provided by the embodiments of the present application " seabed " can be mistakenly identified as " Sea World " by algorithm merely by problem devious, such as above-mentioned example, and " wintersweet " is accidentally known Not Wei " winter jasmine ", " oriental cherry descends slowly and lightly " is mistakenly identified as " snowing ".

For carrying out the scene of scene Recognition to preview image.In some embodiments, as shown in figure 5, in step S403 This method can also include: later

S404, mobile phone 300 adjust the acquisition parameters of camera, so that the tag match of acquisition parameters and the first image.

Wherein, the acquisition parameters of above-mentioned camera include but is not limited to exposure, sensitivity, aperture, white balance, focal length, Exposure Metering, flash lamp etc..Mobile phone 300, can be according to the scene Recognition of preview image after identifying the scene of preview image As a result adjust automatically acquisition parameters improve the regulated efficiency of acquisition parameters without manually adjusting.In addition, mobile phone 300 is automatic The acquisition parameters of adjustment are not compared to being acquisition parameters that very professional user manually adjusts, usually more preferably shooting ginseng Number, is more suitable present preview image, can shoot the photo or video of more high quality.

In one possible implementation, different labels can be in advance from the corresponding relationship of different acquisition parameters It establishes.It, can be from the different label pre-established and pair of different acquisition parameters after the label for determining the first image In should being related to, according to the determining corresponding acquisition parameters of label lookup.

In some embodiments, after completing shooting using acquisition parameters adjusted, acquisition parameters can be restored To initial parameter, or restore to default parameters.Convenient for re-recognizing scene when taking pictures preview next time.Wherein, initial parameter Or default parameters can be the corresponding acquisition parameters of scene that mobile phone is most often shot.For example, what user most often shot is mountains and rivers wind Scape, mobile phone can be by initial parameters or default parameters that " mountains and rivers landscape " corresponding acquisition parameters are set as.Side in this way Formula, can be to avoid frequent adjustment acquisition parameters.

In one possible implementation, adjustment acquisition parameters may include: that the label for the determination that will be found is corresponding Acquisition parameters compared with initial parameter (or default parameters), if the two is identical, without adjustment；If the two is different, will clap It takes the photograph parameter and the corresponding acquisition parameters of the label is adjusted to by initial parameter (or default parameters).

S405, mobile phone 300 detect the shooting instruction of user.

Such as: mobile phone 300 detects the imaging icon on user's point touching screen.Or mobile phone 300 detects other Preset movement, such as volume key is pressed, what this pre-seted dynamic instruction is " taking pictures " or " camera shooting ".

S406, the shooting instruction in response to user, mobile phone 300 are drawn using acquisition parameters shooting preview adjusted Face.

In some embodiments, Cambrian Cambricon instruction set is integrated in the NPU chip of mobile phone 300.NPU chip The treatment process of the label of the first image is determined using Cambricon instruction set acceleration convolutional neural networks.

Wherein, the design principle of Cambricon is:

1) Reduced Instruction Set Computer (the Reduced Instruction based on load-store memory access mode is used Set Computer, RISC).The selection of specific instruction carries out the abstract of calculating level according to the type of workload and obtains.It is right For deep-neural-network (Deep Neural Network, DNN), main calculating and the calculating of control task directed quantity, square Battle array calculates, Scalar operation and branch jump.

2) complicated caching Cache system and associated control logic are not introduced.This has with the workload type of AI algorithm Strong association, for AI algorithm, data locality data locality is not strong, and influence of the cache to performance is unlike routine Calculating task is so big, so the control logic for realizing caching level cache hierarchy is simplified, for being promoted The calculating power dissipation ratio of chip has very big benefit.

3) primary storage of data is calculated using buffer Scratchpad Memory rather than register file.Cause Different from conventional multimedia calculating task for the calculating task of AI algorithm, instructing operated data length is often random length , thus be applied to multimedia instruction optimization SIMD organization (Single Instruction Multiple Data, SIMD it is flexible that register file) is just not so good as Scrathpad Memory.

Wherein, instruction set can be divided into four major class, be to calculate class, logic class, control class and data access class to refer to respectively It enables.It calculates class instruction and provides the support of instruction set primarily directed to the common calculating logic of neural network.Such as matrix and square The multiplication of battle array, matrix is mutually with vector, vector and being multiplied for vector, etc..One feature of this kind of instruction is that instruction is grasped The length for making data is random length, flexibly to support various sizes of matrix and vector.Logic class instruction primarily directed to Amount or matrix data complete logic judgment operation.Such as supporting the condition merge of max-pooling is instructed can be right Multiple groups feature map completes the operation of max-pooling by condition assignment.Control class and the instruction of data access class are compared Simply, it just jumps there is provided branch and the load and write-in of data.

As shown in fig. 6, being a kind of accelerator architecture based on Cambricon instruction set provided by the embodiments of the present application.Its In, the scalar function unit (Scalar Func.Unit) in Fig. 6, phasor function unit (Vector Func.Unit), matrix Function unit (Matrix Func.Unit) instructs after decoding, and it is medium to be first put into tagsort queue (Issue Queue) To.After obtaining action type from scalar register file (Scalar Register File), difference is sent an instruction to Resume module.Control instruction and Scalar operation can be sent directly to scalar function cell processing.Data transfer instruction needs to visit Ask that L1 caches (L1Cache), and the instruction of vector sum matrix correlation is eventually separately sent to phasor function unit and matrix function Unit, the two units are that the operation of vector sum matrix accelerates and specially designs.

Vector sum matrix manipulation instruction in Fig. 6 has used the scratch pad memory (Scratchpad Memorry) in piece. Traditional processor participates in the calculating of processor using the data of the register of regular length, and in neural network, data Often random length, it is using register less real.And traditional framework, register number is very little, is not suitable for vector sum Matrix calculates.Here it is the purposes for using scratch pad memory.Scratch pad memory substitutes traditional register, phasor function unit and Matrix function unit can be calculated with the data of scratch pad memory.In the design of Cambricon, vector scratch pad memory It is 64K, matrix scratch pad memory is 768K.

In addition, also devising 3 directly for phasor function unit and matrix function unit for the access for accelerating scratch pad memory Meet memory access (Direct Memory Access, DMA)；In addition a direct memory access input/output has also been devised. A set of mechanism has also been devised in Cambricon, and scratch pad memory is divided into multiple different bank, to allow while support multiple Input/output interface.

In some embodiments, as shown in fig. 7, after S403, or as shown in figure 8, the application is real after S406 Apply the scene recognition method in example further include:

S701, mobile phone 300 are by the tag update of the first image and the first image into the training set of convolutional neural networks.

S702, mobile phone 300 are according to updated training set re -training convolutional neural networks.

It is understood that mobile phone 300, in order to realize the function of any of the above-described a embodiment, it comprises execute each function It can corresponding hardware configuration and/or software module.Those skilled in the art should be readily appreciated that, in conjunction with disclosed herein Embodiment description each exemplary unit and algorithm steps, the application can be with the combination of hardware or hardware and computer software Form is realized.Some functions is executed in a manner of hardware or computer software driving hardware actually, depends on technical side The specific application and design constraint of case.Professional technician can carry out each specific application to come using distinct methods real Existing described function, but this realization is it is not considered that exceed scope of the present application.

The embodiment of the present application can carry out the division of functional module to mobile phone 300, for example, each function division can be corresponded to Two or more functions can also be integrated in a processing module by each functional module.Above-mentioned integrated module Both it can take the form of hardware realization, can also have been realized in the form of software function module.It should be noted that the application It is schematically that only a kind of logical function partition can have other in actual implementation to the division of module in embodiment Division mode.

For example, in the case where to use the integrated each functional module of model split, as shown in figure 9, implementing for the application A kind of structural schematic diagram for mobile phone that example provides.The mobile phone 300 may include scene Recognition unit 910 and information acquisition unit 920。

Wherein, first image for identification of scene Recognition unit 910, determines the scene Recognition result of the first image.Wherein, It include at least one scene type in scene Recognition result.Information acquisition unit 920 is used to obtain bat when the first image of acquisition Information is taken the photograph, which includes at least: one or more of temporal information, location information, Weather information and temperature information. Scene Recognition unit 910 is also used to, and according to photographing information, the label of the first image is determined from scene Recognition result.Wherein, The label of one image is used to indicate the scene type of the first image.

It is likely to be obtained in structure in one kind, which can also include parameter adjustment unit 930, for knowing in scene Other unit is according to photographing information, after the label for determining the first image in scene Recognition result, adjusts the shooting ginseng of camera Number, so that the tag match of acquisition parameters and the first image.

It should be noted that above-mentioned mobile phone 300 can also include radio circuit.Specifically, mobile phone 300 can pass through radio frequency Circuit carries out sending and receiving for wireless signal.In general, radio circuit includes but is not limited to antenna, at least one amplifier, receives Sender, coupler, low-noise amplifier, duplexer etc..In addition, radio circuit can also by wireless communication and other equipment Communication.The wireless communication can be used any communication standard or agreement, including but not limited to global system for mobile communications, general It is grouped wireless service, CDMA, wideband code division multiple access, long term evolution, Email, short message service etc..

It, can be entirely or partly with computer when being transmitted using software realization data in a kind of optional mode The form of program product is realized.The computer program product includes one or more computer instructions.It loads on computers When with executing the computer program instructions, process or function described in the embodiment of the present application are entirely or partly realized.It is described Computer can be general purpose computer, special purpose computer, computer network or other programmable devices.The computer refers to Order may be stored in a computer readable storage medium, or can to another computer from a computer readable storage medium Storage medium transmission is read, for example, the computer instruction can be from a web-site, computer, server or data center By wired (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, wireless, microwave etc.) mode to Another web-site, computer, server or data center are transmitted.The computer readable storage medium can be meter Any usable medium that calculation machine can access either includes integrated server, the data center etc. of one or more usable mediums Data storage device.The usable medium can be magnetic medium, (such as floppy disk, hard disk, tape), optical medium (such as DVD), Or semiconductor medium (such as solid state hard disk Solid State Disk (SSD)) etc..

The step of method in conjunction with described in the embodiment of the present application or algorithm can realize in a manner of hardware, can also be with It is that the mode of software instruction is executed by processor to realize.Software instruction can be made of corresponding software module, software module RAM memory, flash memory, ROM memory, eprom memory, eeprom memory, register, hard disk, shifting can be stored on In the storage medium of dynamic hard disk, CD-ROM or any other form well known in the art.A kind of illustrative storage medium coupling It is bonded to processor, to enable a processor to from the read information, and information can be written to the storage medium.When So, storage medium is also possible to the component part of processor.Pocessor and storage media can be located in ASIC.In addition, should ASIC can be located in detection device.Certainly, pocessor and storage media can also be used as discrete assembly and be present in detection device In.

Through the above description of the embodiments, it is apparent to those skilled in the art that, for description It is convenienct and succinct, only the example of the division of the above functional modules, in practical application, can according to need and will be upper It states function distribution to be completed by different functional modules, i.e., the internal structure of device is divided into different functional modules, to complete All or part of function described above.

In several embodiments provided herein, it should be understood that disclosed user equipment and method, Ke Yitong Other modes are crossed to realize.For example, Installation practice described above is only illustrative, for example, the module or unit Division, only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or group Part may be combined or can be integrated into another device, or some features can be ignored or not executed.Another point, it is shown Or the mutual coupling, direct-coupling or communication connection discussed can be through some interfaces, device or unit it is indirect Coupling or communication connection can be electrical property, mechanical or other forms.

The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit The component shown can be a physical unit or multiple physical units, it can and it is in one place, or may be distributed over Multiple and different places.Some or all of unit therein can be selected to realize this embodiment scheme according to the actual needs Purpose.

It, can also be in addition, each functional unit in each embodiment of the application can integrate in one processing unit It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list Member both can take the form of hardware realization, can also realize in the form of software functional units.

If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent product When, it can store in a read/write memory medium.Based on this understanding, the technical solution of the embodiment of the present application is substantially The all or part of the part that contributes to existing technology or the technical solution can be in the form of software products in other words It embodies, which is stored in a storage medium, including some instructions are used so that an equipment (can be list Piece machine, chip etc.) or processor (processor) execute each embodiment the method for the application all or part of the steps. And storage medium above-mentioned includes: that USB flash disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), arbitrary access are deposited The various media that can store program code such as reservoir (Random Access Memory, RAM), magnetic or disk.

The above, the only specific embodiment of the application, but the protection scope of the application is not limited thereto, it is any Change or replacement within the technical scope of the present application should all be covered within the scope of protection of this application.Therefore, this Shen Protection scope please should be based on the protection scope of the described claims.

Claims (24)

1. a kind of scene recognition method, which is characterized in that the described method includes:

It identifies the first image, determines the scene Recognition of the first image as a result, including at least one in the scene Recognition result A scene type；

Obtain photographing information when acquisition the first image, the photographing information includes at least: temporal information, location information, One or more of Weather information and temperature information；

According to the photographing information, the label of the first image, the first image are determined from the scene Recognition result Label be used to indicate the scene type of the first image.

2. the method according to claim 1, wherein at least one described scene type, includes at least: in image Image background information, the corresponding season information of image, in the reference object information of the corresponding Weather information of image and image It is one or more.

3. method according to claim 1 or 2, which is characterized in that at least one described scene type is according to described first The descending sequence of matching degree of image and each scene type.

4. according to the method described in claim 3, it is characterized in that, described according to the photographing information, from the scene Recognition As a result the label of the first image is determined in, comprising:

According to the matching degree and at least one described scene type of the photographing information and at least one scene type Descending sequence determines the label of the first image.

5. method according to claim 1-4, which is characterized in that the method is applied to include at neural network Manage the electronic equipment of unit NPU chip；

The identification the first image determines the scene Recognition result of the first image, comprising:

The first image is identified by the NPU chip, determines the scene Recognition result of the first image.

6. according to the method described in claim 5, referring to it is characterized in that, being integrated with Cambrian Cambricon in the NPU chip Enable collection；The NPU chip accelerates to determine the scene Recognition result of the first image using Cambrian Cambricon instruction set Process.

7. method according to claim 5 or 6, which is characterized in that the first image is the camera shooting of the electronic equipment The preview image of head acquisition.

8. method according to claim 1-6, which is characterized in that the first image is stored picture； Alternatively, the first image is the picture obtained from other equipment.

9. the method according to the description of claim 7 is characterized in that according to the photographing information, from the scene Recognition knot After the label for determining the first image in fruit, the method also includes:

The acquisition parameters of the camera are adjusted, so that the tag match of the acquisition parameters and the first image.

10. -9 described in any item methods according to claim 1, which is characterized in that be integrated with convolutional Neural in the NPU chip Network, the method also includes:

By the tag update of the first image and the first image into the training set of the convolutional neural networks；

According to convolutional neural networks described in updated training set re -training.

11. a kind of scene Recognition device, which is characterized in that the scene Recognition device includes:

Scene Recognition unit, the first image, determines the scene Recognition of the first image as a result, the scene Recognition for identification It as a result include at least one scene type in；

Information acquisition unit, for obtaining photographing information when acquisition the first image, the photographing information is included at least: when Between one or more of information, location information, Weather information and temperature information；

The scene Recognition unit is also used to, and according to the photographing information, described first is determined from the scene Recognition result The label of the label of image, the first image is used to indicate the scene type of the first image.

12. device according to claim 11, which is characterized in that at least one described scene type includes at least: image In image background information, the corresponding season information of image, in the reference object information of the corresponding Weather information of image and image One or more.

13. device according to claim 11 or 12, which is characterized in that at least one described scene type is according to described The descending sequence of matching degree of one image and each scene type.

14. device according to claim 13, which is characterized in that the scene Recognition unit according to the photographing information, The label of the first image is determined from the scene Recognition result, comprising:

The scene Recognition unit is according to the matching degree of the photographing information and at least one scene type and described The descending sequence of at least one scene type, determines the label of the first image.

15. the described in any item devices of 1-14 according to claim 1, which is characterized in that the scene Recognition unit includes nerve Network processing unit NPU；

The scene Recognition unit identifies the first image, determines the scene Recognition result of the first image, comprising:

The scene Recognition unit identifies the first image by the NPU chip, determines that the scene of the first image is known Other result.

16. device according to claim 15, which is characterized in that be integrated with Cambrian Cambricon in the NPU chip Instruction set；The NPU chip accelerates to determine the scene Recognition result of the first image using Cambrian Cambricon instruction set Process.

17. device according to claim 15 or 16, which is characterized in that the scene Recognition device further include: camera, The first image is the preview image of the camera acquisition.

18. the described in any item devices of 1-16 according to claim 1, which is characterized in that the first image is stored figure Piece；Alternatively, the first image is the picture obtained from other equipment.

19. device according to claim 17, which is characterized in that described device further include:

Parameter adjustment unit is used in the scene Recognition unit according to the photographing information, from the scene Recognition result After the label for determining the first image, the acquisition parameters of the camera are adjusted, so that the acquisition parameters and described the The tag match of one image.

20. the described in any item devices of 1-19 according to claim 1, which is characterized in that be integrated with convolution mind in the NPU chip Through network, the scene Recognition unit is also used to:

By the tag update of the first image and the first image into the training set of the convolutional neural networks；

According to convolutional neural networks described in updated training set re -training.

21. a kind of user equipment (UE), which is characterized in that the UE includes: scene Recognition device, and the scene Recognition device is used for Execute such as the described in any item scene recognition methods of claim 1-10.

22. a kind of user equipment (UE), which is characterized in that the UE includes:

Memory, for storing computer program code, the computer program code includes instruction；

Radio circuit, for carrying out sending and receiving for wireless signal；

Processor realizes such as the described in any item scene recognition methods of claim 1-10 for executing described instruction.

23. a kind of computer readable storage medium, computer executed instructions are stored on the computer readable storage medium, institute It states when computer executed instructions circuit processed executes and realizes such as the described in any item scene recognition methods of claim 1-10.

24. a kind of chip system, which is characterized in that the chip system includes processor, memory, is stored in the memory There is instruction；When described instruction is executed by the processor, such as the described in any item scene recognition methods of claim 1-10 are realized.