WO2020077522A1 - 一种识别环境场景的方法、芯片、终端 - Google Patents

一种识别环境场景的方法、芯片、终端 Download PDF

Info

Publication number
WO2020077522A1
WO2020077522A1 PCT/CN2018/110390 CN2018110390W WO2020077522A1 WO 2020077522 A1 WO2020077522 A1 WO 2020077522A1 CN 2018110390 W CN2018110390 W CN 2018110390W WO 2020077522 A1 WO2020077522 A1 WO 2020077522A1
Authority
WO
WIPO (PCT)
Prior art keywords
scene
terminal
user
image data
algorithm model
Prior art date
Application number
PCT/CN2018/110390
Other languages
English (en)
French (fr)
Inventor
刘沛
孙忠
李大伟
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to PCT/CN2018/110390 priority Critical patent/WO2020077522A1/zh
Priority to CN201880091730.3A priority patent/CN111903113A/zh
Publication of WO2020077522A1 publication Critical patent/WO2020077522A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/725—Cordless telephones
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Definitions

  • the present application relates to the field of communications, and more specifically, to a method, chip, and terminal for identifying environmental scenarios.
  • AI artificial intelligence technology With the development of AI artificial intelligence technology, the application of AI technology in terminal devices is becoming more and more extensive, making the functions of terminal devices more and more intelligent. For example, with the popularization of AI technology in terminal devices, the functions of terminal devices in the fields of perception, image processing, audio processing, and language processing are becoming more and more powerful.
  • AI artificial intelligence is integrated in the software system.
  • the user needs to take a picture of the environmental scene actively before the environmental scene around the user To identify.
  • the environment scene function in the terminal device does not continue to be constantly on to sense whether the user environment has changed, and its user experience is not good.
  • Embodiments of the present application provide a method, chip, and terminal for identifying an environmental scenario, which can enable the terminal to be independent of specific actions or environmental factors in a low power consumption mode. It can sense the user's environment and scene changes in real time, and can actively provide users with more natural human-computer interaction and a better user experience.
  • a method for identifying an environmental scene includes: a terminal obtains image data in real time through a low-power camera, the low-power camera is always on; the terminal analyzes a user based on the image data Whether the surrounding environment scene is a target environment scene; the terminal determines that the user's surrounding environment scene changes to the target environment scene; and the terminal enables a work mode corresponding to artificial intelligence AI.
  • the low-power camera in the embodiment of the present application may be normally open at a specific frame rate, and may collect image data around the terminal in real time.
  • a normally-open low-power camera can be used as an infrastructure and continuously collect data around the terminal to provide a hardware foundation for the terminal device to implement AI artificial intelligence technology autonomously.
  • the terminal invokes an environment recognition algorithm model to analyze whether the surrounding environment scene of the user is the target environment scene according to the image data.
  • the embodiment of the present application does not specifically limit the specific implementation manner of establishing the environment recognition algorithm model.
  • the algorithm model can be constructed based on a deep learning neural network, and supervised learning training can be performed through large-scale labeled environmental picture data, so that the trained algorithm model can be deployed in the terminal .
  • the algorithm since a deep learning neural network is used to construct an environment recognition algorithm model, the algorithm does not require manual design and extraction of environment features, and the neural network automatically learns features from a large amount of training data.
  • the terminal determines that the surrounding environment scene of the user changes to the scene to be muted.
  • the terminal determines that the surrounding environment scene of the user changes to the scene to be mute; the terminal adjusts the working mode to a working mode of vibration or mute.
  • the environment recognition algorithm model is a neural network algorithm model
  • the neural network algorithm model is obtained from supervised learning training based on large-scale target environment scene data
  • the terminal inputs the image data into the neural network algorithm model, and the neural network algorithm model calls a corresponding operator in the AI operator library to analyze whether the user's surrounding environment data in the image data is target environment data .
  • the AI operator library is solidified in the hardware of the terminal.
  • the neural network algorithm model calls the corresponding operator in the AI operator library through a hardware accelerator, and analyzes the user's surroundings in the image data Whether the environmental data is the target environmental data.
  • the terminal is in a sleep state before the terminal starts a work mode corresponding to artificial intelligence AI.
  • a chip for recognizing an environmental scenario includes: a coprocessor and a main processor, the coprocessor is connected to the main processor,
  • the coprocessor is used to perform the following operations: obtain image data in real time through a low power camera, the low power camera is connected to the coprocessor, the low power camera is always on; the user is analyzed based on the image data Whether the surrounding environment scene is a target environment scene; determining that the surrounding environment scene of the user changes to the target environment scene, and sending an AI message to the main processor.
  • the main processor is used to perform the following operation: start the working mode corresponding to the AI according to the received AI message.
  • the coprocessor may be connected to the main processor, and the low-power camera may be connected to the coprocessor through the driver layer of the coprocessor.
  • the coprocessor can call the AI algorithm model (also called environment recognition algorithm) integrated in the coprocessor according to the image data collected by the normally-open low-power camera to analyze and determine whether the user's surrounding environment scene has changed.
  • AI algorithm model also called environment recognition algorithm
  • the coprocessor can report the environmental scene recognition result to the main controller when it is determined that there is an environmental scene change.
  • the coprocessor may generate an AI message at the AI application layer, and may report the AI message to the main controller.
  • the coprocessor can collect user image data in real time through a normally-open low-power camera, and can recognize whether the environmental scene in the image is the set target environmental scene through a computer vision algorithm according to the image data.
  • the AI recognition result is reported to the main controller.
  • the system can be woken up and can adjust the working mode of the terminal device accordingly according to the AI recognition result.
  • the application scene of the terminal device can be automatically adjusted to a conference mode (for example, vibration or mute mode) ).
  • a conference mode for example, vibration or mute mode
  • the application scene of the terminal device may be automatically adjusted to the driving mode (For example, ring pattern)
  • the main controller system when there is no service, the main controller system normally sleeps and stands by, and enters a low power consumption mode.
  • the main controller system is woken up.
  • the main controller can switch the terminal device to the corresponding working mode according to different target scenarios.
  • the target environment scene is a scene to be muted.
  • the coprocessor includes: an AI engine module, an environment recognition algorithm model, an AI algorithm library module, and an AI application layer module,
  • the AI engine module is used to: according to the image data collected by the low-power camera, call the environment recognition algorithm model to analyze whether the surrounding environment scene of the user in the image data is a scene requiring silence.
  • the environment recognition algorithm model is used to: call a corresponding operator in the AI algorithm library to analyze whether the surrounding environment scene of the user in the image data is a scene to be mute, and determine that the image data changes from outdoor scene data to When it is necessary to mute the scene scene data, the environmental scene recognition result is reported to the AI application layer.
  • the AI application layer is used to report the AI message to the main controller according to the environment scene recognition result.
  • the main processor is specifically configured to: according to the received AI message, adjust the working mode to a working mode of vibration or mute.
  • the environment recognition algorithm model is a neural network algorithm model
  • the neural network algorithm model is obtained from supervised learning training based on large-scale target environment scene data .
  • the AI algorithm library is solidified in the hardware of the coprocessor.
  • the coprocessor further includes: a hardware accelerator module configured to call an AI algorithm library module on the environment recognition algorithm model and analyze the image data The process of whether the surrounding environment of the user is a scene that needs to be silenced is accelerated.
  • the main controller is in a sleep state before the main controller turns on the corresponding terminal working mode according to the received AI message.
  • a terminal including: a coprocessor, a main processor, and a low power camera, the coprocessor is connected to the main processor, and the low power camera and the coprocessor Connected,
  • the coprocessor is used to perform the following operations: obtain image data in real time through a low power camera, the low power camera is connected to the coprocessor, the low power camera is always on; the user is analyzed based on the image data Whether the surrounding environment scene is a target environment scene; determining that the surrounding environment scene of the user changes to the target environment scene, and sending an AI message to the main processor.
  • the main processor is configured to: enable the working mode corresponding to the AI according to the received AI message.
  • the coprocessor is specifically configured to: according to the image data, call an environment recognition algorithm model to analyze whether the surrounding environment scene of the user is the target environment scene.
  • the target environment scene is a scene to be muted.
  • the coprocessor is specifically configured to: according to the image data, call the environment recognition algorithm model to analyze whether the user's surrounding environment scene in the image data Scenes that need to be muted.
  • the environment recognition algorithm model is a neural network algorithm model
  • the neural network algorithm model is based on supervised learning training based on large-scale target environment scene data
  • the coprocessor is specifically used to: input the image data into the neural network algorithm model, and the neural network algorithm model calls a corresponding operator in the AI operator library to analyze the image data in the Whether the user's surrounding environment data is the target environment data
  • the AI operator library is solidified in the hardware of the coprocessor.
  • the coprocessor further includes: a hardware accelerator module configured to call an AI algorithm library module on the environment recognition algorithm model and analyze the image data The process of whether the surrounding environment of the user is a scene that needs to be silenced is accelerated.
  • the main controller before the main controller turns on the corresponding terminal working mode according to the received AI message, the main controller is in a sleep state.
  • a terminal including:
  • the acquisition module is used to acquire image data in real time through a low-power camera, which is always on.
  • the analysis module is used for analyzing whether the surrounding scene of the user is the target environment scene according to the image data.
  • the determining module is configured to determine that the surrounding environment scene of the user changes to the target environment scene.
  • the processing module is used to enable the work mode corresponding to artificial intelligence AI.
  • the analysis module is specifically configured to: based on the image data, call an environment recognition algorithm model to analyze whether the surrounding environment scene of the user is the target environment scene.
  • the target environment scene is a scene to be muted.
  • the environment recognition algorithm model is a neural network algorithm model
  • the neural network algorithm model is obtained from supervised learning training based on large-scale target environment scene data
  • the analysis module is specifically used to: input the image data into the neural network algorithm model, and the neural network algorithm model calls a corresponding operator in the AI operator library to analyze the user in the image data Whether the surrounding environmental data is the target environmental data.
  • the AI operator library is solidified in the hardware of the terminal.
  • the analysis module is specifically configured to: the neural network algorithm model calls a corresponding operator in the AI operator library through a hardware accelerator, and analyzes the Whether the user's surrounding environment data in the image data is target environment data.
  • the sleep mode is enabled before the work mode corresponding to artificial intelligence AI is turned on.
  • a computer-readable storage medium including a computer program, when the computer program runs on a computer, the computer is executed as described in the first aspect or any implementation manner of the first aspect method.
  • a computer program product which, when the computer program product runs on a computer, causes the computer to execute the method as described in the first aspect or any implementation manner of the first aspect.
  • FIG. 1 is a schematic flowchart of a method for identifying a user ’s surrounding environment scene provided by an embodiment of the present application.
  • FIG. 2 is a schematic block diagram of a hardware architecture of a terminal 200 provided by an embodiment of the present application.
  • FIG. 3 is a schematic flowchart of a scene recognition scenario of a terminal device environment provided by an embodiment of the present application.
  • FIG. 4 is a schematic structural diagram of a terminal 400 provided by an embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of a chip 500 for identifying an environmental scene provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of the interface change of the terminal after using the method for recognizing a user ’s surrounding environment scene provided by an embodiment of the present application.
  • the type of the terminal device (which may also be referred to as a terminal) is not specifically limited, and the terminal device may include but is not limited to a mobile station (MS), a mobile phone (mobile telephone) , User equipment (UE), mobile phone (handset), portable equipment (portable), cellular phone, cordless phone, session initiation protocol (SIP) phone, wireless local loop (wireless local loop, WLL) ) Stations, personal digital processing (personal digital assistant, PDA), radio frequency identification (RFID) terminal devices for logistics, handheld devices with wireless communication functions, computing devices or other devices connected to wireless modems, in-vehicle devices , Wearable devices, Internet of Things, terminal devices in the vehicle network.
  • MS mobile station
  • UE User equipment
  • PDA personal digital assistant
  • RFID radio frequency identification
  • the terminal device may also be a wearable device.
  • Wearable devices can also be referred to as wearable smart devices, which is a general term for applying wearable technology to intelligently design everyday wear and develop wearable devices, such as glasses, gloves, watches, clothing and shoes.
  • a wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only a hardware device, but also realize powerful functions through software support, data interaction, and cloud interaction.
  • Generalized wearable smart devices include full-featured, large-sized, complete or partial functions that do not depend on smartphones, such as smart watches or smart glasses, and only focus on a certain type of application functions, and need to cooperate with other devices such as smartphones Use, such as various smart bracelets and smart jewelry for sign monitoring.
  • Artificial intelligence is a theory, method, technology, and application system that uses digital computers or digital computer-controlled machines to simulate, extend, and expand human intelligence, sense the environment, acquire knowledge, and use knowledge to obtain the best results.
  • artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence.
  • Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machine has the functions of perception, reasoning and decision-making.
  • Research in the field of artificial intelligence includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, basic AI theory, etc.
  • AI artificial intelligence technology With the development of AI artificial intelligence technology, the application of AI technology in terminal devices is becoming more and more extensive, making the functions of terminal devices more and more intelligent. For example, with the popularization of AI technology in terminal devices, the functions of terminal devices in the fields of perception, image processing, audio processing, and language processing are becoming more and more powerful.
  • AI artificial intelligence is integrated in a software system, and the AI artificial intelligence function basically requires a certain action of the user or triggering of other application modules.
  • the corresponding application module will call the corresponding AI artificial intelligence function.
  • the function of AI artificial intelligence in the terminal device will not continue to be normally open, and the AI technology will not be constantly open to perceive changes in the user's surrounding environment scenes, behavior intentions, and environmental changes.
  • the user when a computer vision algorithm is used to identify environmental scenes in an image, the user is required to actively take photographs of the environmental scenes around the user, and then the terminal device will recognize the environmental scenes in the saved pictures.
  • the embodiments of the present application provide a method for identifying a user's surrounding environment scene, which can make the terminal device not dependent on the user's specific operation, can sense the change of the user environment in real time, and can provide the user with the ability to seamlessly perceive the application service
  • the terminal equipment is more intelligent and the man-machine experience is more comfortable.
  • FIG. 1 is a schematic flowchart of a method for identifying a user ’s surrounding environment scene provided by an embodiment of the present application.
  • the method shown in FIG. 1 may include steps 110-140, and steps 110-140 will be described in detail below.
  • Step 110 The terminal acquires image data in real time through a low-power camera.
  • the low-power camera in the embodiment of the present application can be turned on at a specific frame rate, so that image data around the terminal device can be collected in real time, and the collected image data can be reported to the terminal.
  • a normally-open low-power camera can be used as an infrastructure and continuously collect data around the terminal to provide a hardware foundation for the terminal device to implement AI artificial intelligence technology autonomously.
  • Step 120 The terminal analyzes whether the surrounding environment scene of the user is the target scene according to the image data.
  • the terminal may continuously detect the image based on the normally-open low-power camera, and may analyze whether the poor surrounding environment of the user is the target scene according to the collected image.
  • the terminal may analyze whether the surrounding environment scene of the user is a target scene through an environment recognition algorithm.
  • Step 130 The terminal determines that the surrounding environment scene of the user changes to the target environment scene.
  • the terminal may determine whether the detection result of the environment scene has changed according to the state of the surrounding environment scene around the user. That is, the terminal may determine that the surrounding environment scene of the user changes to the target environment scene.
  • the coprocessor in the terminal can generate an AI event message at the AI application layer, and can report the AI event message to the main controller in the terminal.
  • the specific implementation will be described in detail below in conjunction with FIG. 2 and will not be repeated here.
  • Step 140 the terminal starts the work mode corresponding to artificial intelligence AI.
  • the working mode corresponding to the AI turned on by the terminal in the embodiment of the present application can be understood as the application mode corresponding to different environmental scenarios.
  • the application scene corresponding to the terminal is a conference mode, for example, the terminal works in a vibration or silent working mode.
  • the application scenario corresponding to the terminal is a driving mode, for example, the terminal works in a ringtone mode.
  • the terminal may automatically recognize the environment scene around the user based on the continuously detected images around the user, and when determining that the environment scene around the user changes to the target environment scene, turn on artificial intelligence AI correspondence Working mode.
  • the terminal can switch to the corresponding scene mode according to the environment recognition result.
  • the terminal may include a main processor, a coprocessor, and a normally-open low-power camera.
  • the main controller system in the terminal sleeps normally and enters a low power consumption mode.
  • the coprocessor in the terminal reports the AI event message
  • the main controller system in the terminal is woken up.
  • the main controller in the terminal can implement various bright business functions according to product business requirements, or pass event messages to other related business modules, and other business modules complete the final processing.
  • the coprocessor in the terminal can collect user image data in real time through a normally-open low-power camera, and can recognize whether the environmental scene in the image is the set target environmental scene through a computer vision algorithm according to the image data.
  • the AI recognition result is reported to the main controller.
  • the system can be woken up and can adjust the working mode of the terminal device accordingly according to the AI recognition result.
  • the application scene of the terminal device may be automatically adjusted to a conference mode (for example, vibration or mute mode ).
  • a conference mode for example, vibration or mute mode
  • the set target environment scene is an outdoor environment (such as an in-car environment)
  • the application scene of the terminal device may be automatically adjusted to the driving mode ( For example, ring pattern).
  • the embodiment of the present application does not specifically limit the specific implementation manner of establishing the environment recognition algorithm model.
  • the algorithm model can be constructed based on a deep learning neural network, and supervised learning training can be performed through large-scale labeled environmental picture data, so that the trained algorithm model can be deployed in co-processing ⁇ ⁇ .
  • the algorithm since a deep learning neural network is used to construct an environment recognition algorithm model, the algorithm does not require manual design and extraction of environment features, and the neural network automatically learns features from a large amount of training data.
  • FIG. 2 is a schematic block diagram of a hardware architecture of a terminal 200 provided by an embodiment of the present application.
  • the hardware architecture of the terminal 200 shown in FIG. 2 may include a main processor 210, a coprocessor 220, and a low-power camera 230.
  • Coprocessor 220 Integrated AI capabilities, can continuously run in low power mode to detect changes in user environment scenes.
  • the coprocessor 220 is connected to the main processor 210, and upon detecting a change in the environment around the user, triggers the wake-up of the main controller 210 by reporting an AI event message to the main processor 210.
  • Main processor 210 When there is no service, the main controller 210 system can perform a normal sleep standby state and enter a low power consumption mode. After receiving the AI event message sent by the coprocessor 220, after the main processor 210 is woken up, it receives the event reported by the coprocessor 220 and triggers the corresponding business scene function.
  • Low power consumption normally-on camera 230 connected to the coprocessor 220 through a peripheral chip software interface (driver) provided by the coprocessor 220, and providing a data source for the coprocessor 220 to process AI services.
  • driver peripheral chip software interface
  • the software system of the coprocessor 220 may be a real-time operating system (RTOS).
  • RTOS real-time operating system
  • the results of its processing can control the production process or respond quickly to the processing system within the prescribed time, and schedule all available resources to complete real-time tasks. And control all real-time tasks coordinated operation of the operating system, fast response and high reliability.
  • the RTOS system of the coprocessor 220 may include: a kernel 221, a framework layer (framework layer) 222, and an APP application layer 223.
  • the kernel 221 includes a peripheral driver module 2211, a hardware acceleration module 2212, and an AI operator library module 2213.
  • Framework layer 222 includes: AI application management module 2221, AI algorithm management module 2222, and AI algorithm model 2223.
  • the APP application layer 223 includes: an AI application layer module 2231, an AI engine module 2232, and an AI model management module 2233.
  • Peripheral driver module 2211 It can provide a software interface for various peripheral chips. For example, a normally-open low-power camera 230 may be connected, and the low-power camera 230 may provide a hardware basis for the coprocessor 220 to perceive the user's surrounding environment scene intentions or environmental changes. The coprocessor 220 can analyze the characteristics of the user's actions and surrounding environment according to the image data collected by the low-power camera 230, and provide a data source for the coprocessor 220 to process AI services.
  • the terminal may acquire the image data in real time through the normally-open low-power camera 230 connected to the peripheral drive module 2211.
  • the peripheral devices connected to the peripheral drive module 2211 may also include, but are not limited to: sensors (which can be used to recognize user actions), normally-open low-power microphones (which can be used to analyze user voice Etc.), position sensors (for example, global positioning system (GPS), wireless local area network (WIFI), modem) can be used to provide user's location information.
  • sensors which can be used to recognize user actions
  • normally-open low-power microphones which can be used to analyze user voice Etc.
  • position sensors for example, global positioning system (GPS), wireless local area network (WIFI), modem
  • AI application management module 2221 It can classify the data reported by the peripheral drive module 2211. For example, the received data is divided into image categories, video categories, audio categories, etc., so as to call AI algorithm models 2223 of different categories for analysis and processing.
  • AI engine module 2232 it can be responsible for scheduling and coordinating the AI algorithm model 2223 for operation. Since there are multiple AI algorithm models 2223 running at the same time, the scheduling management control of the AI engine module 2232 can ensure the orderly operation of the software to the greatest extent.
  • AI algorithm management module 2222 responsible for algorithm management, according to the different types of data reported by the AI application management module 2221, the corresponding AI algorithm model can be selected from a plurality of running AI algorithm models 2223 for analysis.
  • AI algorithm model 2223 It can be a set of algorithm features that conform to the image and sound of certain services. For example, when recognizing whether the surrounding environment scene of the user is a target scene, the AI algorithm model 2223 may be a set conforming to the target environment scene. The AI algorithm model 2223 can be trained through large-scale image data. After the training is completed, an algorithm model can be generated, and the algorithm model can be run by the corresponding AI operator to automatically recognize the scene surrounding the user.
  • the co-processing 220 in the terminal may receive the image data reported by the normally-open low-power camera 230, and the AI application management module 2221 may identify the user's surrounding environment scene in the analysis image according to the data to be processed.
  • the corresponding environment recognition algorithm is called through the AI engine module 2232 to analyze whether the surrounding environment scene of the user in the collected image is the target environment scene (for example, a scene to be muted).
  • AI algorithm model 2223 may be integrated into the software system by default, or may be updated into the coprocessor 220 through the main controller 210, which is not specifically limited in the embodiment of the present application.
  • the main controller 210 may also optimize the AI algorithm model 2223. For example, positioning information such as GPS / WIFI / modem can be used to comprehensively judge the results of the AI algorithm model 2223 to improve the accuracy of the AI algorithm model 2223.
  • the AI model management module 2233 can modify certain features in the AI algorithm model 2223.
  • AI operator library module 2213 The AI engine module 2232 can run the AI model management module 2233 for environment recognition by calling the operator in the AI operator library module 2213. Due to the limited resources of the coprocessor 220, the AI operator library module 2213, which designs a large number of mathematical calculations, can be solidified in hardware, and most of the AI operators can be implemented by the hardware, which can avoid the high processor load generated by the software implementation operator.
  • the interface of the hardware curing operator may be provided by the kernel 221 to the AI model management module 2233 for use.
  • the curing of the AI operator library module 2213 in the hardware may be to write the software on the coprocessor chip, and the programmed software may be run through the coprocessor chip.
  • Software curing is to make software on silicon chips (so-called firmware) to realize software functions, so that the complexity of the operating system and language processing is shared by both hardware and software.
  • the AI operator library module 2213 is fixed on the hardware of the coprocessor.
  • the operation of the software curing can increase the operation speed of the entire system, improve reliability, reduce costs, and facilitate mass production and standardization.
  • Hardware acceleration module 2212 It is possible to accelerate the process of running the AI model management module 2233 by calling the operator in the AI operator library module 2213 to the AI engine module 2232 through the acceleration mode. It can ensure that the AI engine module 2232 can quickly call the operators in the AI operator library module 2213 in real time, and provide capability interfaces for various AI algorithms in the framework layer 222AI model management module 2233.
  • AI application layer module 2231 It can be located in the APP application layer 223, and can implement various continuous and normally open AI applications in the APP application layer 223 according to the scene requirements of the terminal device business design.
  • the AI application layer module 2231 can call various algorithms to obtain AI recognition results of various peripherally connected devices, and can report the corresponding AI event message to the main controller 210. If the main controller 210 is in the dormant state, it can perform secondary processing on the AI event message after being awakened.
  • the AI application management module 2221 reports the environmental recognition result to the AI application layer Module 2231.
  • the AI application layer module 2231 will form an environment recognition event message and report the environment recognition event message to the AI event message manager 212 in the main controller 210.
  • the system architecture of the main processor 210 is described in detail below.
  • Main processor 210 responsible for running various applications of the terminal device, including UI human-computer interaction interface, and cloud interaction. When there is no business, the main controller system sleeps normally and enters a low power consumption mode.
  • the main processor 210 may include: AI local (AI) 211, AI event message manager (AI) service 212, application (application, APP) 213, APP 214, APP 215.
  • AI AI local
  • AI AI event message manager
  • AI local (AI) 211 The AI event message reported by the coprocessor 220 can be received, and the main controller 210 is woken up.
  • the AI algorithm model 2223 optimized by the main controller 210 may also be sent to the AI engine module 2232 of the coprocessor 220, and the AI engine module 2232 may update the AI algorithm model 2223 through the AI model management module 2233.
  • AI event message manager (AI) service 212 It can receive AI event messages reported by AI native 211, and manage the AI capability interface of terminal equipment in a unified manner, and provide AI application program interfaces (application interprograme, API) for each business module. According to product business needs, realize various bright business functions. For example, different highlight business functions can be implemented according to different applications (APP213 or APP214 or APP215).
  • the main controller 210 is woken up.
  • the main controller 210 may automatically change the working mode of the terminal device when it is determined that the environment surrounding the user detected in the image reported by the power consumption camera 230 changes to the target environment scene.
  • the terminal device Taking the target environment scene as the scene to be mute, if the environment recognition result received by the main controller 210 is that the user scene is switched to the scene to be mute, the terminal device automatically adjusts to the conference mode (for example, vibration or mute mode).
  • the conference mode for example, vibration or mute mode
  • the terminal device automatically adjusts to the driving mode (for example, ringtone mode).
  • the driving mode for example, ringtone mode
  • AI service 212 can also transfer the data to the cloud to complete a low-power service processing mode in which terminal devices and the cloud are combined.
  • the main frequency of the coprocessor is low
  • the AI operators involved in a large number of mathematical operations are integrated in a hardware-hardened manner
  • the peripheral devices are low-power devices, which can be operated in a low-power mode Normally open and run AI perception capabilities, so that the terminal device can not rely on specific actions, can sense changes in user actions or changes in the surrounding environment scene.
  • the target environment scene is a scene to be muted or a scene in a car, which is only described as an example of two environment recognition.
  • the corresponding application scene modes are conference mode and driving mode.
  • the environment recognition scenes of the embodiments of the present application are not limited to the above two categories, and the other environment recognition scenes can be applied to the method and chip for recognizing environmental scenes provided by the embodiments of the present application.
  • FIG. 3 The specific implementation of the terminal shown in FIG. 2 for real-time sensing of changes in the user ’s environment scene will be described in detail below with reference to the target environment scene in FIG. 3 as a scene requiring silence.
  • FIG. 3 is only to help those skilled in the art to understand the embodiments of the present application, but not to limit the embodiments of the application to specific numerical values or specific scenarios of the illustrated examples.
  • Those skilled in the art can obviously make various equivalent modifications or changes according to the example of FIG. 3 given in the text, and such modifications and changes also fall within the scope of the embodiments of the present application.
  • FIG. 3 is a schematic flowchart of a scene recognition scenario of a terminal device environment provided by an embodiment of the present application.
  • the method shown in FIG. 3 may include steps 310-355, and steps 310-355 will be described in detail below.
  • Step 310 Start.
  • Step 315 The low-power camera collects images.
  • the co-processor 220 can continuously collect images around the terminal device through the connected normally-open low-power camera 230, and can report the collected image data to the AI application management module 2221.
  • Step 320 The coprocessor calls the environment recognition algorithm model to detect whether the environment scene around the user is a scene scene to be muted.
  • the co-processing 220 can receive the image data reported by the normally-open low-power camera 230, and the AI application management module 2221 can identify the user's surrounding environment scene in the analysis image according to the data to be processed, and call it through the AI engine module 2232
  • the corresponding environment recognition algorithm analyzes whether the surrounding environment scene of the user in the collected image is a scene scene to be muted.
  • the environment recognition algorithm model can be constructed based on a deep learning neural network, and supervised learning training can be performed through large-scale labeled environment picture data, so that the trained environment recognition algorithm model can be deployed in a coprocessor in.
  • the AI application management module 2221 may perform step 325.
  • the AI application management module 2221 may re-execute step 315.
  • Step 325 Compared with the previous state, determine whether the environmental scene detection result has changed.
  • the environment recognition algorithm After the environment recognition algorithm detects that the user's surrounding environment scene in the image is a scene that needs to be mute, it can be compared with the previous state to see whether the environmental scene detection result has changed.
  • the AI application management module 2221 may perform step 330 if compared with the previous state.
  • the AI application management module 2221 can re-execute the steps if compared with the previous state 315.
  • Step 330 The coprocessor reports the environment identification message to the main controller.
  • the AI application management module 2221 reports the environmental recognition results to the main controller 210.
  • the AI application management module 2221 in the coprocessor 220 may report the environment recognition result to the AI application layer module 2231. After obtaining the environment recognition result, the AI application layer module 2231 will form an environment recognition event message and report the environment recognition event message to the AI event message manager 212 in the main controller 210.
  • Step 335 The main controller is woken up.
  • the AI event message manager 212 in the main controller 210 wakes up after receiving the environment recognition event message sent by the AI application layer module 2231.
  • Step 340 The main controller judges whether to switch the terminal to the conference mode.
  • the main controller 210 may determine whether the surrounding environment scene of the user detected in the image reported by the power consumption camera 230 changes to a scene scene that needs to be muted.
  • step 345 may be performed.
  • step 350 may be performed.
  • Step 345 The main controller automatically sets the terminal to the conference mode.
  • the main controller 210 may automatically adjust the terminal device to the conference mode (for example, vibration or silent mode) when it is determined that the scene surrounding the user detected in the image reported by the power consumption camera 230 changes to the conference mode.
  • the conference mode for example, vibration or silent mode
  • Step 350 The main controller does not change the terminal working mode.
  • the main controller 210 may not change the working mode of the terminal device when determining that the surrounding environment scene of the user detected in the image reported by the power consumption camera 230 has not changed to the conference mode.
  • Step 355 End.
  • the terminal can automatically detect changes in the user's surrounding environment, and the interface of the terminal changes from 610 to interface 620 automatically.
  • the terminal automatically adjusts to the conference mode (for example, the vibration mode, as 630 in FIG. 6).
  • the terminal device can collect user image data in real time through a normally-open low-power camera, and can autonomously run AI perception capabilities.
  • the terminal device can automatically recognize the environment and switch to the corresponding scene mode (for example, conference mode), so that the terminal device is more intelligent and the human-machine experience is more comfortable.
  • FIG. 4 is a schematic structural diagram of a terminal 400 provided by an embodiment of the present application.
  • the terminal 400 may include: an acquisition module 410, an analysis module 420, a determination module 430, and a processing module 440.
  • the above modules are described in detail below.
  • the obtaining module 410 is used to obtain image data in real time through a low-power camera, which is always on.
  • the analysis module 420 is configured to analyze whether the surrounding scene of the user is a target environment scene according to the image data.
  • the determining module 430 is configured to determine that the surrounding environment scene of the user changes to the target environment scene.
  • the processing module 440 is used to: enable a work mode corresponding to artificial intelligence AI.
  • the analysis module 420 is specifically configured to: according to the image data, call an environment recognition algorithm model to analyze whether the surrounding environment scene of the user is the target environment scene.
  • the target environment scene is a scene to be muted.
  • the environment recognition algorithm model is a neural network algorithm model
  • the neural network algorithm model is obtained from supervised learning training based on large-scale target environment scene data
  • the analysis module 420 is specifically configured to: input the image data into the neural network algorithm model, and the neural network algorithm model calls a corresponding operator in the AI operator library to analyze the user in the image data Whether the surrounding environmental data is the target environmental data.
  • the AI operator library is solidified in the hardware of the terminal.
  • the analysis module 420 is specifically configured to: the neural network algorithm model calls a corresponding operator in the AI operator library through a hardware accelerator, and analyzes the image data Whether the surrounding environment data of the user is the target environment data.
  • it is in a sleep state before the work mode corresponding to artificial intelligence AI is turned on.
  • FIG. 5 is a schematic structural diagram of a chip 500 for identifying an environmental scene provided by an embodiment of the present application.
  • the chip 500 may include: a main processor 510 and a coprocessor 520.
  • the coprocessor 520 is used to perform the following operations: obtain image data in real time through a low-power camera, the low-power camera is always on; based on the image data, analyze whether the user's surrounding environment scene is the target environment scene; determine Said the surrounding environment scene of the user changes to the target environment scene, and sends an AI message to the main processor.
  • the main processor 510 is configured to: enable the working mode corresponding to the AI according to the received AI message.
  • the coprocessor 520 may be connected to the main processor 510, and the low-power camera may be connected to the coprocessor 520.
  • the coprocessor 520 can call the AI algorithm model (also called environment recognition algorithm) integrated in the coprocessor 520 according to the image data collected by the normally open low-power camera to analyze and determine whether the user's surrounding environment scene has changed .
  • AI algorithm model also called environment recognition algorithm
  • the coprocessor 520 may report the environmental scene recognition result to the main controller 510 if it is determined that there is an environmental scene change. As an example, the coprocessor 520 may generate an AI message at the AI application layer, and may report the AI message to the main controller 510.
  • an environment recognition algorithm model is invoked to analyze whether the surrounding environment scene of the user is the target environment scene.
  • the coprocessor 520 is specifically configured to: the target environment scene is a scene to be muted.
  • the coprocessor 520 includes: an AI engine module, an environment recognition algorithm model, an AI algorithm library module, and an AI application layer module,
  • the AI engine module is used to: according to the image data collected by the low-power camera, call the environment recognition algorithm model to analyze whether the surrounding environment scene of the user in the image data is a scene requiring silence.
  • the environment recognition algorithm model is used to: call a corresponding operator in the AI algorithm library to analyze whether the surrounding environment scene of the user in the image data is a scene to be mute, and determine that the image data changes from outdoor scene data to When it is necessary to mute the scene scene data, the environmental scene recognition result is reported to the AI application layer.
  • the AI application layer is used to report the AI message to the main controller according to the environment scene recognition result.
  • the environment recognition algorithm model is a neural network algorithm model
  • the neural network algorithm model is obtained from supervised learning training based on large-scale target environment scene data.
  • the AI algorithm library is solidified in the hardware of the coprocessor.
  • the coprocessor 520 further includes: a hardware accelerator module, configured to call an AI algorithm library module on the environment recognition algorithm model, and analyze the user's surrounding environment scene in the image data Whether to accelerate the process of muting scenes.
  • a hardware accelerator module configured to call an AI algorithm library module on the environment recognition algorithm model, and analyze the user's surrounding environment scene in the image data Whether to accelerate the process of muting scenes.
  • the main controller 510 before the main controller 510 turns on the corresponding terminal working mode according to the received AI message, the main controller 510 is in a sleep state.
  • An embodiment of the present application also provides a computer-readable storage medium, including a computer program, which, when the computer program is run on a terminal, causes the terminal to perform the methods described in steps 110-140 and the like.
  • An embodiment of the present application also provides a computer program product, which, when the computer program product runs on a terminal, causes the terminal to perform the method described in steps 110-140 and the like.
  • the term "article of manufacture" used in the embodiments of the present application encompasses a computer program accessible from any computer-readable device, carrier, or medium.
  • the computer-readable medium may include, but is not limited to: magnetic storage devices (for example, hard disks, floppy disks, or magnetic tapes, etc.), optical disks (for example, compact discs (CD), digital universal discs (digital discs, digital discs, DVD)) Etc.), smart cards and flash memory devices (for example, erasable programmable read-only memory (EPROM), cards, sticks or key drives, etc.).
  • various storage media described herein may represent one or more devices and / or other machine-readable media for storing information.
  • machine-readable medium may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.
  • the disclosed system, device, and method may be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the units is only a division of logical functions.
  • there may be other divisions for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
  • the function is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium.
  • the technical solutions of the embodiments of the present application may essentially be part of or contribute to the prior art or the technical solutions may be embodied in the form of software products, and the computer software products are stored in a storage medium , Including several instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program codes .

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • User Interface Of Digital Computer (AREA)
  • Telephone Function (AREA)

Abstract

一种识别用户周边环境场景的方法、芯片、终端，该方法包括：终端通过常开的低功耗摄像头获取图像数据（110）；所述终端根据所述图像数据，调用环境识别算法分析用户周边环境场景是否为目标场景（120）；所述终端确定所述用户周边环境场景变化为所述目标环境场景（130）；所述终端开启人工智能AI对应的工作模式（140）。该技术方案中终端可以持续常开AI人工智能能力，实时感知用户周围环境场景的变化，从而可以主动为用户提供更自然的人机交互和更好的用户体验。

Description

一种识别环境场景的方法、芯片、终端 技术领域
本申请涉及通信领域，并且更具体地，涉及一种识别环境场景的方法、芯片、终端。
背景技术
随着AI人工智能技术的发展，AI技术在终端设备上的应用越来越广泛，使得终端设备的功能越来越智能化。例如，随着AI技术在终端设备上的普及推广，终端设备在感知领域、图像处理领域、音频处理领域、语言处理领域等方面的功能越来越强大。
现有技术中，AI人工智能集成在软件系统中，在机器视觉领域，通过计算机视觉算法技术识别图像中的环境场景时，需要用户主动对环境场景进行拍照时候，才会对用户周边的环境场景进行识别。该终端设备中的环境场景功能不会持续常开来感知用户环境是否发生改变，其用户体验度不佳。
因此，如何使得终端设备能够主动、实时感知用户环境场景改变，提高用户体验度成为亟需要解决的问题。
发明内容
本申请实施例提供一种识别环境场景的方法、芯片、终端，可以使得终端在低功耗模式下，不依赖特定的动作或环境因素。能够实时感知用户环境场景改变，可以主动为用户提供更自然的人机交互和更好的用户体验。
第一方面，提供了一种用于识别环境场景的方法，该方法包括：终端通过低功耗摄像头实时获取图像数据，所述低功耗摄像头一直开启；所述终端根据所述图像数据分析用户周边环境场景是否为目标环境场景；所述终端确定所述用户周边环境场景变化为所述目标环境场景；所述终端开启人工智能AI对应的工作模式。
本申请实施例中的低功耗的摄像头可以以特定帧率常开，可以实时采集终端周边的图像数据。
应理解，常开的低功耗摄像头可以作为基础设施，并且持续不断的采集终端周围的数据，为终端设备实现自主运行AI人工智能技术提供硬件基础。
结合第一方面，在第一方面的某些实现方式中，所述终端根据所述图像数据，调用环境识别算法模型分析用户周边环境场景是否为目标环境场景。
本申请实施例对建立环境识别算法模型的具体实现方式不做具体限定。可选地，在一些实施例中，可以基于深度学习神经网络来构建该算法模型，可以通过大规模已标注的环境图片数据来进行监督学习训练，从而可以将训练所得的算法模型部署在终端中。
本申请实施例中，由于采用深度学习神经网络来构建环境识别算法模型，该算法不需要人工设计和提取环境特征，而由神经网络从大量训练数据中自动学习特征。
结合第一方面，在第一方面的某些实现方式中，所述终端确定所述用户周边环境场景 变化为所述需静音场景。
结合第一方面，在第一方面的某些实现方式中，所述终端确定所述用户周边环境场景变化为所述需静音场景；所述终端将工作模式调整为震动或静音的工作模式。
结合第一方面，在第一方面的某些实现方式中，所述环境识别算法模型为神经网络算法模型，所述神经网络算法模型为根据大规模的目标环境场景数据进行监督学习训练而成的，
所述终端将所述图像数据输入到所述神经网络算法模型中，所述神经网络算法模型调用AI算子库中对应的算子分析所述图像数据中的用户周边环境数据是否为目标环境数据。
结合第一方面，在第一方面的某些实现方式中，所述AI算子库固化在所述终端的硬件中。
结合第一方面，在第一方面的某些实现方式中，所述神经网络算法模型通过硬件加速器调用所述AI算子库中对应的算子，并分析所述所述图像数据中的用户周边环境数据是否为目标环境数据。
结合第一方面，在第一方面的某些实现方式中，在所述终端开启人工智能AI对应的工作模式之前，所述终端处于休眠状态。
第二方面，提供了一种用于识别环境场景的芯片，该芯片包括：协处理器、主处理器，所述协处理器与所述主处理器相连，
所述协处理器用于执行以下操作：通过低功耗摄像头实时获取图像数据，所述低功耗摄像头与所述协处理器相连，所述低功耗摄像头一直开启；根据所述图像数据分析用户周边环境场景是否为目标环境场景；确定所述用户周边环境场景变化为所述目标环境场景，向所述主处理器发送AI消息。
所述主处理器用于执行以下操作：根据接收到的所述AI消息开启所述AI对应的工作模式。
本申请实施例中协处理器可以与主处理器相连，低功耗摄像头可以通过协处理器的驱动层与协处理相连。
协处理器可以根据常开的低功耗摄像头采集到的图像数据，调用集成在协处理器中的AI算法模型(也可以称为环境识别算法)分析判断用户的周围环境场景是否发生变化。
协处理器可以在判断到有环境场景发生变化的情况下，将环境场景识别结果上报给主控制器。作为一个示例，协处理器可以在AI应用层生成AI消息，并可以向主控制器上报该AI消息。
作为一个示例，协处理器可以通过常开的低功耗摄像头实时采集用户图像数据，并可以根据该图像数据通过计算机视觉算法识别图像中的环境场景是否为设定的目标环境场景。在判断到用户周边环境场景变化为目标环境场景的情况下，将AI识别结果上报给主控制器。主控制器在接收到协处理器上报的AI识别结果之后，该系统可以被唤醒，并可以根据AI识别结果相应的调整终端设备的工作模式。例如，如果设定的目标环境场景为需静音场景，当检测到用户周围环境由其他环境场景变化为需静音场景时，可以自动将终端设备的应用场景调整为会议模式(例如，震动或静音模式)。又如，如果设定的目标环境场景为室外环境(例如车内环境)，当检测到用户周围环境由其他环境场景变化为车内 环境时，可以自动将终端设备的应用场景调整为驾驶模式(例如，铃声模式)
本申请实施例中在没有业务时，主控制器系统正常休眠待机，进入低功耗模式。当协处理器上报环境场景变化的消息后，主控制器系统被唤醒。主控制器可以根据目标场景的不同，将终端设备切换到对应的工作模式。
结合第二方面，在第二方面的某些实现方式中，所述目标环境场景为需静音场景。
结合第二方面，在第二方面的某些实现方式中，所述协处理器包括：AI引擎模块、环境识别算法模型、AI算法库模块，AI应用层模块，
所述AI引擎模块用于：根据所述低功耗摄像头采集的图像数据，调用所述环境识别算法模型分析所述图像数据中的用户周边环境场景是否为需静音场景。
所述环境识别算法模型用于：调用所述AI算法库中对应的算子分析所述图像数据中的用户周边环境场景是否为需静音场景，并在确定所述图像数据由室外场景数据变化为需静音场景场景数据时，将环境场景识别结果上报至所述AI应用层。
所述AI应用层用于：根据所述环境场景识别结果，向所述主控制器上报所述AI消息。
结合第二方面，在第二方面的某些实现方式中，所述主处理器具体用于：根据接收到的所述AI消息，将工作模式调整为震动或静音的工作模式。
结合第二方面，在第二方面的某些实现方式中，所述环境识别算法模型为神经网络算法模型，所述神经网络算法模型为根据大规模的目标环境场景数据进行监督学习训练而成的。
结合第二方面，在第二方面的某些实现方式中，所述AI算法库固化在所述协处理器的硬件中。
结合第二方面，在第二方面的某些实现方式中，所述协处理器还包括：硬件加速器模块，用于对所述环境识别算法模型调用AI算法库模块，并分析所述图像数据中的用户周边环境场景是否为需静音场景的过程进行加速。
结合第二方面，在第二方面的某些实现方式中，在所述主控制器根据接收到的所述AI消息开启相应的终端工作模式之前，所述主控制器处于休眠状态。
第三方面，提供了一种终端，包括：协处理器、主处理器、低功耗摄像头，所述协处理器与所述主处理器相连，所述低功耗摄像头与所述协处理器相连，
所述协处理器用于执行以下操作：通过低功耗摄像头实时获取图像数据，所述低功耗摄像头与所述协处理器相连，所述低功耗摄像头一直开启；根据所述图像数据分析用户周边环境场景是否为目标环境场景；确定所述用户周边环境场景变化为所述目标环境场景，向所述主处理器发送AI消息。
所述主处理器用于：根据接收到的所述AI消息开启所述AI对应的工作模式。
结合第三方面，在第三方面的某些实现方式中，所述协处理器具体用于：根据所述图像数据，调用环境识别算法模型分析用户周边环境场景是否为目标环境场景。
结合第三方面，在第三方面的某些实现方式中，所述目标环境场景为需静音场景。
结合第三方面，在第三方面的某些实现方式中，所述协处理器具体用于：根据所述图像数据，调用所述环境识别算法模型分析所述图像数据中的用户周边环境场景是否为需静音场景。
结合第三方面，在第三方面的某些实现方式中，所述环境识别算法模型为神经网络算 法模型，所述神经网络算法模型为根据大规模的目标环境场景数据进行监督学习训练而成的，所述协处理器具体用于：将所述图像数据输入到所述神经网络算法模型中，所述神经网络算法模型调用AI算子库中对应的算子分析所述所述图像数据中的用户周边环境数据是否为目标环境数据
结合第三方面，在第三方面的某些实现方式中，所述AI算子库固化在所述协处理器的硬件中。
结合第三方面，在第三方面的某些实现方式中，所述协处理器还包括：硬件加速器模块，用于对所述环境识别算法模型调用AI算法库模块，并分析所述图像数据中的用户周边环境场景是否为需静音场景的过程进行加速。
结合第三方面，在第三方面的某些实现方式中，在所述主控制器根据接收到的所述AI消息开启相应的终端工作模式之前，所述主控制器处于休眠状态。
第四方面，提供了一种终端，包括：
获取模块，用于通过低功耗摄像头实时获取图像数据，所述低功耗摄像头一直开启。
分析模块，用于根据所述图像数据分析用户周边场景是否为目标环境场景。
确定模块，用于确定所述用户周边环境场景变化为所述目标环境场景。
处理模块，用于开启人工智能AI对应的的工作模式。
结合第四方面，在第四方面的某些实现方式中，分析模块具体用于：根据所述图像数据，调用环境识别算法模型分析用户周边环境场景是否为目标环境场景。
结合第四方面，在第四方面的某些实现方式中，所述目标环境场景为需静音场景。
结合第四方面，在第四方面的某些实现方式中，所述环境识别算法模型为神经网络算法模型，所述神经网络算法模型为根据大规模的目标环境场景数据进行监督学习训练而成的，所述分析模块具体用于：将所述图像数据输入到所述神经网络算法模型中，所述神经网络算法模型调用AI算子库中对应的算子分析所述所述图像数据中的用户周边环境数据是否为目标环境数据。
结合第四方面，在第四方面的某些实现方式中，所述AI算子库固化在所述终端的硬件中。
结合第四方面，在第四方面的某些实现方式中，所述分析模块具体用于：所述神经网络算法模型通过硬件加速器调用所述AI算子库中对应的算子，并分析所述所述图像数据中的用户周边环境数据是否为目标环境数据。
结合第四方面，在第四方面的某些实现方式中，在开启人工智能AI对应的工作模式之前处于休眠状态。
第五方面，提供了一种计算机可读存储介质，包括计算机程序，当该计算机程序在计算机上运行时，使得该计算机如执行第一方面或第一方面的任意一种实现方式中所述的方法。
第六方面，提供了一种计算机程序产品，当该计算机程序产品在计算机上运行时，使得该计算机执行如第一方面或第一方面任意一种实现方式中所述的方法。
附图说明
图1是本申请实施例提供的一种用于识别用户周边环境场景的方法的示意性流程图。
图2是本申请实施例提供的一种终端200的硬件架构示意性框图。
图3是本申请实施例提供的一种终端设备环境场景识别场景的示意性流程图。
图4是本申请实施例提供的一种终端400的示意性结构图。
图5是本申请实施例提供的一种用于识别环境场景的芯片500的示意性结构图。
图6是使用本申请实施例提供的用于识别用户周边环境场景的方法之后终端的界面变化示意图。
具体实施方式
下面将结合附图，对本申请实施例中的技术方案进行描述。
应理解，本申请实施例中对提及的终端设备(也可以简称为终端)的类型不做具体限定，终端设备可以包括但不限于移动台(mobile station，MS)、移动电话(mobile telephone)、用户设备(user equipment，UE)、手机(handset)、便携设备(portable equipment)、蜂窝电话、无绳电话、会话启动协议(session initiation protocol，SIP)电话、无线本地环路(wireless local loop，WLL)站、个人数字处理(personal digital assistant，PDA)、物流用的射频识别(radio frequency identification，RFID)终端设备，具有无线通信功能的手持设备、计算设备或连接到无线调制解调器的其它设备、车载设备、可穿戴设备、物联网、车辆网中的终端设备。
作为示例而非限定，在本申请实施例中，该终端设备还可以是可穿戴设备。可穿戴设备也可以称为穿戴式智能设备，是应用穿戴式技术对日常穿戴进行智能化设计、开发出可以穿戴的设备的总称，如眼镜、手套、手表、服饰及鞋等。可穿戴设备即直接穿在身上，或是整合到用户的衣服或配件的一种便携式设备。可穿戴设备不仅仅是一种硬件设备，更是通过软件支持以及数据交互、云端交互来实现强大的功能。广义穿戴式智能设备包括功能全、尺寸大、可不依赖智能手机实现完整或者部分的功能，例如：智能手表或智能眼镜等，以及只专注于某一类应用功能，需要和其它设备如智能手机配合使用，如各类进行体征监测的智能手环、智能首饰等。
人工智能(artificial intelligence，AI)是利用数字计算机或者数字计算机控制的机器模拟、延伸和扩展人的智能，感知环境、获取知识并使用知识获得最佳结果的理论、方法、技术及应用系统。换句话说，人工智能是计算机科学的一个分支，它企图了解智能的实质，并生产出一种新的能以人类智能相似的方式作出反应的智能机器。人工智能也就是研究各种智能机器的设计原理与实现方法，使机器具有感知、推理与决策的功能。人工智能领域的研究包括机器人，自然语言处理，计算机视觉，决策与推理，人机交互，推荐与搜索，AI基础理论等。
随着AI人工智能技术的发展，AI技术在终端设备上的应用越来越广泛，使得终端设备的功能越来越智能化。例如，随着AI技术在终端设备上的普及推广，终端设备在感知领域、图像处理领域、音频处理领域、语言处理领域等方面的功能越来越强大。
现有技术中，AI人工智能集成在软件系统中，该AI人工智能功能基本上都需要用户的某一个动作或其他应用模块的触发。现有技术的终端设备中在有业务需要时，对应的应用模块才会调用相应的AI人工智能功能。该终端设备中的AI人工智能的功能不会持续常开，不会通过AI技术持续常开来感知用户周边环境场景、行为意图、环境改变等变化。
例如，在使用计算机视觉算法识别图像中的环境场景时，需要用户主动对该用户周边的环境场景进行拍照，然后该终端设备才会对保存的图片中的环境场景进行识别。
因此，现有技术中，在使用计算机视觉算法识别图像中的环境场景时，需要用户主动对该用户周边的环境场景进行拍照，该终端设备不能主动、实时感知用户周边的环境场景状态，其用户体验度不佳。
本申请实施例提供了一种用于识别用户周边环境场景的方法，可以使得终端设备不依赖用户的特定操作，可以实时感知用户环境的变化，可以为用户提供无缝感知应用业务的能力，使得终端设备更智能化，人机体验更舒适。
图1是本申请实施例提供的一种用于识别用户周边环境场景的方法的示意性流程图。图1所示的方法可以包括步骤110-140，下面对步骤110-140进行详细描述。
步骤110，终端通过低功耗摄像头实时获取图像数据。
本申请实施例中的低功耗的摄像头可以以特定帧率一直开启，从而可以实时采集终端设备周边的图像数据，并可以将采集到的图像数据上报至终端。
应理解，常开的低功耗摄像头可以作为基础设施，并且持续不断的采集终端周围的数据，为终端设备实现自主运行AI人工智能技术提供硬件基础。
步骤120，终端根据所述图像数据，分析用户周边环境场景是否为目标场景。
本申请实施例中终端可以根据常开的低功耗摄像头不断的检测到的图像，并可以根据采集到的图像分析用户周围环境差场景是否为目标场景。作为一个示例，终端可以通过环境识别算法分析用户周边环境场景是否为目标场景。
步骤130，终端确定所述用户周边环境场景变化为目标环境场景。
终端可以在步骤120中判断到用户周边环境场景是否为目标场景之后，可以再根据用户周围之前的环境场景状态，判断该环境场景检测结果是否发生变化。也就是说，终端可以确定所述用户周边环境场景变化为目标环境场景。
具体地，终端中的协处理器可以在AI应用层生成AI事件消息，并可以向终端中的主控制器上报该AI事件消息。下面会结合图2对该具体实现方式进行详细描述，此处不再赘述。
步骤140，终端开启人工智能AI对应的工作模式。
本申请实施例中的终端开启的AI对应的工作模式可以理解为不同的环境场景对应的应用模式。作为一个示例，环境场景为需静音场景时，终端对应的应用场景为会议模式，例如，该终端工作在震动或静音工作模式。作为另一个示例，环境场景为车内环境时，终端对应的应用场景为驾驶模式，例如，该终端工作在铃声模式。
本申请实施例中，终端可以根据持续不断检测到的用户周围的图像，自动识别用户周围的环境场景，并在确定所述用户周边环境场景变化为所述目标环境场景时，开启人工智能AI对应的工作模式。该终端可以根据环境识别结果切换到对应的场景模式。为用户提供无缝感知应用业务的能力，使得终端设备更智能化，人机体验更舒适。
本申请实施例中终端可以包括：主处理器、协处理器以及常开的低功耗摄像头。
在没有业务时，终端中的主控制器系统正常休眠待机，进入低功耗模式。当终端中的协处理器上报AI事件消息后，终端中的主控制器系统被唤醒。终端中的主控制器可以根据产品业务需求，实现各种亮点业务功能，或者将事件消息传递给其他相关的业务模块， 由其他业务模块完成最终的处理。
作为一个示例，终端中的协处理器可以通过常开的低功耗摄像头实时采集用户图像数据，并可以根据该图像数据通过计算机视觉算法识别图像中的环境场景是否为设定的目标环境场景。在判断到用户周边环境场景变化为目标环境场景的情况下，将AI识别结果上报给主控制器。终端中的主控制器在接收到协处理器上报的AI识别结果之后，该系统可以被唤醒，并可以根据AI识别结果相应的调整终端设备的工作模式。例如，如果设定的目标环境场景为需静音场景，当检测到用户周围环境由其他环境场景变化为需静音场景时，可以自动将终端设备的应用场景调整为会议模式(例如，震动或静音模式)。又如，如果设定的目标环境场景为室外环境(例如车内环境)，当检测到用户周围环境由其他环境场景变化为车内环境时，可以自动将终端设备的应用场景调整为驾驶模式(例如，铃声模式)。下面会结合图2-图3对终端设备自动识别周围环境场景并调整终端设备的工作模式的具体实现方式进行详细描述，此处不再赘述。
本申请实施例对建立环境识别算法模型的具体实现方式不做具体限定。可选地，在一些实施例中，可以基于深度学习神经网络来构建该算法模型，可以通过大规模已标注的环境图片数据来进行监督学习训练，从而可以将训练所得的算法模型部署在协处理器中。
本申请实施例中，由于采用深度学习神经网络来构建环境识别算法模型，该算法不需要人工设计和提取环境特征，而由神经网络从大量训练数据中自动学习特征。
下面结合图2详细描述本申请实施例中终端中的主处理器和协处理器协同处理，从而实现终端可以自主运行AI感知能力，实时感知用户环境场景的变化的具体实现方式。
图2是本申请实施例提供的一种终端200的硬件架构示意性框图。图2所示的终端200的硬件架构可以包括主处理器210、协处理器220、低功耗摄像头230。
协处理器220：集成了AI能力，可以以低功耗模式持续运行检测用户环境场景变化。协处理器220与主处理器210相连，当检测到用户周围的环境发生变化后，通过向主处理器210上报AI事件消息触发唤醒主控制器210。
主处理器210：在没有业务时，主控制器210系统可以进行正常休眠待机状态，进入低功耗模式。当接收到协处理器220发送的AI事件消息之后，主处理器210被唤醒后，接收协处理器220上报的事件，触发相应的业务场景功能。
低功耗常开摄像头230：通过协处理器220提供的外围芯片软件接口(驱动)与协处理器220相连，为协处理器220处理AI业务提供了数据来源。
下面对协处理器220的系统架构进行详细描述。
协处理器220的软件系统可以是一个实时操作系统(real time operating system，RTOS)。当外界事件或数据产生时，能够接受并以足够快的速度予以处理。其处理的结果又能在规定的时间之内来控制生产过程或对处理系统做出快速响应，调度一切可利用的资源完成实时任务。并控制所有实时任务协调一致运行的操作系统，响应速度快，可靠性高。
协处理器220的RTOS系统可以包括：内核(kernel)221、框架层(framework层)222、APP应用层223。
内核(kernel)221包括：外设驱动模块2211、硬件加速模块2212、AI算子库模块2213。
framework层222包括：AI应用管理模块2221、AI算法管理模块2222、AI算法模型2223。
APP应用层223包括：AI应用层模块2231、AI引擎模块2232、AI模型管理模块2233。
下面对上述几种模块进行详细描述。
外设驱动模块2211：可以为连接有各类外围芯片提供软件接口。例如，可以连接常开的低功耗摄像头230，该低功耗摄像头230可以为协处理器220感知用户周边环境场景意图或环境变化提供了硬件基础。协处理器220可以根据低功耗摄像头230采集到的图像数据，分析用户的动作以及周围环境等特征，为协处理器220处理AI业务提供了数据来源。
具体的，终端可以通过外设驱动模块2211连接的常开的低功耗摄像头230，实时获取图像数据。
可选地，在一些实施例中，连接在外设驱动模块2211的外围器件还可以包括但不限于：传感器(可以用于识别用户动作)、常开的低功耗麦克风(可以用于分析用户语音等特征)、位置传感器(例如，全球定位系统(global postem system，GPS)、无线局域网(wireless fidelity，WIFI)、调制解调器(modem)，可以用于提供用户的位置信息。
AI应用管理模块2221：可以对外设驱动模块2211上报的数据进行分类。例如将接收到的数据分为图像类、视频类、音频类等，以便于调用不同类别的AI算法模型2223进行分析处理。
AI引擎模块2232：可以负责调度、协调AI算法模型2223进行运算。由于同时有多个AI算法模型2223运行，AI引擎模块2232的调度管理控制可以最大限度的保证软件有序运行。
AI算法管理模块2222：负责算法管理，可以根据AI应用管理模块2221上报的不同类别的数据，从多个运行的AI算法模型2223中选择出对应的AI算法模型进行分析。
AI算法模型2223：可以是符合某些业务的图像、声音的算法特征的集合。例如，在进行识别用户周围环境场景是否为目标场景时，该AI算法模型2223可以是符合目标环境场景的集合。AI算法模型2223可以通过大规模的图像数据进行训练，训练完成之后生成算法模型，并可以由对应的AI算子运行该算法模型自动进行用户周围环境场景识别。
具体的，终端中的协处理220可以在接收到常开的低功耗摄像头230上报的图像数据之后，AI应用管理模块2221可以根据需要处理的数据为分析图像中的用户周边环境场景进行识别，通过AI引擎模块2232调用对应的环境识别算法分析采集到的图像中用户周边环境场景是否为目标环境场景(例如，需静音场景)。
需要说明的是，AI算法模型2223可以默认集成在软件系统中，也可以通过主控制器210更新到协处理器220中，本申请实施例对此不做具体限定。
AI模型管理模块2233：在一些实施例中，主控制器210还可以对AI算法模型2223进行优化。例如，可以使用GPS/WIFI/modem等定位信息对AI算法模型2223的结果进行综合判断，以提高AI算法模型2223的准确率。AI模型管理模块2233可以对AI算法模型2223中的某些特征进行修改。
AI算子库模块2213：AI引擎模块2232可以通过调用AI算子库模块2213中的算子来运行AI模型管理模块2233进行环境识别。由于协处理器220资源有限，可以将设计大 量数学计算的AI算子库模块2213固化在硬件中，可以由硬件实现AI的大部分算子，可以避免软件实现算子产生的高处理器负荷。硬件固化算子的接口可以由内核(kernel)221提供接口给AI模型管理模块2233使用。
应理解，AI算子库模块2213固化在硬件中(软件固化)可以是将软件写到协处理器芯片上，可以通过协处理器芯片来运行烧写上的软件。软件固化即把软件制做在硅片(就是所谓固件)上来实现软件功能，使操作系统和语言处理的复杂性由软硬件双方分担。
本申请实施例中，将AI算子库模块2213固化在协处理器的硬件上，该软件固化的操作可以提高整个系统的操作速度，改善可靠性，降低成本，便于大规模生产和实现标准化。
硬件加速模块2212：可以通过加速模式，对AI引擎模块2232调用AI算子库模块2213中的算子来运行AI模型管理模块2233的过程进行加速。可以保证AI引擎模块2232快速实时的调用AI算子库模块2213中的算子，为framework层222AI模型管理模块2233中的各类AI算法提供能力接口。
AI应用层模块2231：可以位于APP应用层223，可以按照终端设备业务设计的场景需求，在APP应用层223实现各种持续常开的AI应用。AI应用层模块2231可以调用到各类算法得到外围连接的各类器件的AI识别结果之后，并可以将对应的AI事件消息上报给主控制器210。如果主控制器210是处于休眠状态，可以在被唤醒之后，对该AI事件消息进行二次处理。
具体的，终端在低功耗摄像头230上报的图像由其他环境场景(例如，车内)变化为现在采集到的图像的目标环境场景时，AI应用管理模块2221将环境识别结果上报给AI应用层模块2231。AI应用层模块2231在得到环境识别结果之后，就会形成环境识别事件消息，并将该环境识别事件消息上报给主控制器210中的AI事件消息管理器212。
下面对主处理器210的系统架构进行详细描述。
主处理器210：负责运行终端设备的各类应用，包括UI人机交互界面，和云端交互等。在没有业务时，主控制器系统正常休眠待机，进入低功耗模式。
主处理器210可以包括：AI本地(AI native)211、AI事件消息管理器(AI service)212、应用(application，APP)213、APP 214、APP 215。
AI本地(AI native)211：可以接收协处理器220上报的AI事件消息，主控制器210被唤醒。还可以将主控制器210优化后的AI算法模型2223发送至协处理器220的AI引擎模块2232，AI引擎模块2232可以通过AI模型管理模块2233将对AI算法模型2223进行更新。
AI事件消息管理器(AI service)212：可以接收AI native 211上报的AI事件消息，并统一管理终端设备的AI能力接口，为各个业务模块提供AI应用程序界面(application program interfae，API)。根据产品业务需求，实现各种亮点业务功能。例如，可以根据不同的应用(APP 213或APP 214或APP 215)，实现不同的亮点业务功能。
具体的，主控制器210中的AI事件消息管理器212在接收到AI应用层模块2231发送的环境识别事件消息之后，该主控制器210被唤醒。主控制器210可以在判断功耗摄像头230上报的图像中检测到的用户周围环境场景变化为目标环境场景的情况下，可以自动更改终端设备的工作模式。
以目标环境场景为需静音场景，如果主控制器210接收到的环境识别结果为用户场景 切换到了需静音场景，则该终端设备自动调整为会议模式(例如，震动或静音模式)。
以目标环境场景为车内或室外为例，如果主控制器210接收到的环境识别结果为用户场景切换到了车内或室外则该终端设备自动调整为驾驶模式(例如，铃声模式)。
可选地，在一些实施例中，如果需要大数据处理，AI service 212还可以将数据传递到云端，完成终端设备和云结合的低功耗业务处理模式。
本申请实施例中，协处理器运行的主频较低，涉及的大量数学运算的AI算子是以硬件固化的方式集成，并且外围的器件为低功耗器件，可以在低功耗的模式下常开并运行AI感知能力，使得终端设备可以不依赖特定的动作，能够感知用户的动作变化或周围环境场景的变化。
本申请实施例中以目标环境场景为需静音场景场景或车内场景，仅仅是作为两个环境识别的实例进行描述。其对应的应用场景的模式分别为会议模式、驾驶模式。但是本申请实施例的环境识别的场景不限于上述两种类别，其他的环境识别的场景均可以适用本申请实施例提供的用于识别环境场景的方法以及芯片。
下面结合图3中的目标环境场景为需静音场景场景作为示例，对图2所示的终端进行实时感知用户环境场景的变化的具体实现方式进行详细描述。应注意，图3的例子仅仅是为了帮助本领域技术人员理解本申请实施例，而非要将申请实施例限制于所示例的具体数值或具体场景。本领域技术人员根据文所给出的图3的例子，显然可以进行各种等价的修改或变化，这样的修改和变化也落入本申请实施例的范围内。
图3是本申请实施例提供的一种终端设备环境场景识别场景的示意性流程图。图3所示的方法可以包括步骤310-355，下面对步骤310-355进行详细描述。
步骤310：开始。
步骤315：低功耗摄像头采集图像。
协处理器220可以通过连接的常开低功耗摄像头230不断采集终端设备周围的图像，并可以将采集到的图像数据上报给AI应用管理模块2221。
步骤320：协处理器调用环境识别算法模型，检测用户周围的环境场景是否是需静音场景场景。
协处理220可以在接收到常开的低功耗摄像头230上报的图像数据之后，AI应用管理模块2221可以根据需要处理的数据为分析图像中的用户周边环境场景进行识别，通过AI引擎模块2232调用对应的环境识别算法分析采集到的图像中用户周边环境场景是否为需静音场景场景。
本申请实施例中可以基于深度学习神经网络来构建该环境识别算法模型，可以通过大规模已标注的环境图片数据来进行监督学习训练，从而可以将训练所得的环境识别算法模型部署在协处理器中。
如果识别出图像中的用户周边环境场景为需静音场景场景，则AI应用管理模块2221可以执行步骤325。
如果识别出图像中的用户周边环境场景不是需静音场景场景，则AI应用管理模块2221可以重新执行步骤315。
步骤325：与之前状态相比，判断环境场景检测结果是否发生变化。
可以在环境识别算法检测出图像中用户的周边环境场景为需静音场景场景之后，可以 和之前的状态对比，看环境场景检测结果是否发生变化。
如果对比之前的状态，发现低功耗摄像头230上报的图像由其他环境场景(例如，车内)变化为现在采集到的图像的需静音场景场景时，则AI应用管理模块2221可以执行步骤330。
如果对比之前的状态，发现低功耗摄像头230上报的图像没有由其他环境场景(例如，车内)变化为现在采集到的图像的需静音场景场景时，则AI应用管理模块2221可以重新执行步骤315。
步骤330：协处理器将环境识别消息上报给主控制器。
在低功耗摄像头230上报的图像由其他环境场景(例如，车内)变化为现在采集到的图像的需静音场景场景时，AI应用管理模块2221将环境识别结果上报给主控制器210。
具体的，协处理器220中的AI应用管理模块2221可以将环境识别结果上报给AI应用层模块2231。AI应用层模块2231在得到环境识别结果之后，就会形成环境识别事件消息，并将该环境识别事件消息上报给主控制器210中的AI事件消息管理器212。
步骤335：主控制器被唤醒。
主控制器210中的AI事件消息管理器212在接收到AI应用层模块2231发送的环境识别事件消息之后，该主控制器210被唤醒。
步骤340：主控制器判断是否将终端切换为会议模式。
主控制器210可以在被唤醒之后，可以判断功耗摄像头230上报的图像中检测到的用户周围环境场景是否变化为需静音场景场景。
如果主控制器210判断功耗摄像头230上报的图像中检测到的用户周围环境场景变化为需静音场景场景的情况下，可以执行步骤345。
如果主控制器210判断功耗摄像头230上报的图像中检测到的用户周围环境场景没有变化为需静音场景场景的情况下，可以执行步骤350。
步骤345：主控制器自动设置该终端为会议模式。
主控制器210可以在判断功耗摄像头230上报的图像中检测到的用户周围环境场景变化为会议模式的情况下，该终端设备自动调整为会议模式(例如，震动或静音模式)。
步骤350：主控制器不改变终端工作模式。
主控制器210可以在判断功耗摄像头230上报的图像中检测到的用户周围环境场景没有变化为会议模式的情况下，可以不改变终端设备的工作模式。
步骤355：结束。
参见图6，当用户从室外场景进入到需静音场景场景之后，该终端可以自动检测到用户周边环境的变化，终端的界面由610自动变成了界面620。从图6中可以看出，当用户从室外场景进入到需静音场景场景之后，终端自动调整为会议模式(例如，震动模式，如图6中的630)。
本申请实施例中，终端设备可以通过常开的低功耗摄像头实时采集用户图像数据，并可自主运行AI感知能力。终端设备可以自动识别环境并切换到对应的场景模式(例如，会议模式)，使得终端设备更智能化，人机体验更舒适。
上文结合图1至图3，详细描述了本发明实施例提供的一种用于识别用户周边环境场景的方法，下面详细描述本申请实施例的装置实施例。应理解，方法实施例的描述与装置 实施例的描述相互对应，因此，未详细描述的部分可以参见前面方法实施例。
图4是本申请实施例提供的一种终端400的示意性结构图。终端400可以包括：获取模块410、分析模块420、确定模块430、处理模块440。下面对上述几种模块进行详细描述。
获取模块410用于：通过低功耗摄像头实时获取图像数据，所述低功耗摄像头一直开启。
分析模块420用于：根据所述图像数据，分析用户周边场景是否为目标环境场景。
确定模块430用于：确定所述用户周边环境场景变化为所述目标环境场景。
处理模块440用于：开启人工智能AI对应的的工作模式。
可选地，在一些实施例中，所述分析模块420具体用于：根据所述图像数据，调用环境识别算法模型分析用户周边环境场景是否为目标环境场景。
可选地，在一些实施例中，所述目标环境场景为需静音场景。
可选地，在一些实施例中，所述环境识别算法模型为神经网络算法模型，所述神经网络算法模型为根据大规模的目标环境场景数据进行监督学习训练而成的，
所述分析模块420具体用于：将所述图像数据输入到所述神经网络算法模型中，所述神经网络算法模型调用AI算子库中对应的算子分析所述所述图像数据中的用户周边环境数据是否为目标环境数据。
可选地，在一些实施例中，所述AI算子库固化在所述终端的硬件中。
可选地，在一些实施例中，所述分析模块420具体用于：所述神经网络算法模型通过硬件加速器调用所述AI算子库中对应的算子，并分析所述所述图像数据中的用户周边环境数据是否为目标环境数据。
可选地，在一些实施例中，在开启人工智能AI对应的工作模式之前处于休眠状态。
图5是本申请实施例提供的一种用于识别环境场景的芯片500的示意性结构图，芯片500可以包括：主处理器510、协处理器520。
所述协处理器520用于执行以下操作：通过低功耗摄像头实时获取图像数据，所述低功耗摄像头一直开启；根据所述图像数据，分析用户周边环境场景是否为目标环境场景；确定所述用户周边环境场景变化为目标环境场景，向所述主处理器发送AI消息。
所述主处理器510用于：根据接收到的所述AI消息开启该AI对应的工作模式。
本申请实施例中协处理器520可以与主处理器510相连，低功耗摄像头可以与协处理器520相连。
协处理器520可以根据常开的低功耗摄像头采集到的图像数据，调用集成在协处理器520中的AI算法模型(也可以称为环境识别算法)分析判断用户的周围环境场景是否发生变化。
协处理器520可以在判断到有环境场景发生变化的情况下，将环境场景识别结果上报给主控制器510。作为一个示例，协处理器520可以在AI应用层生成AI消息，并可以向主控制器510上报该AI消息。
可选地，在一些实施例中，根据所述图像数据，调用环境识别算法模型分析用户周边环境场景是否为目标环境场景。
可选地，在一些实施例中，协处理器520具体用于：所述目标环境场景为需静音场景。
可选地，在一些实施例中，所述协处理器520包括：AI引擎模块、环境识别算法模型、AI算法库模块，AI应用层模块，
所述AI引擎模块用于：根据所述低功耗摄像头采集的图像数据，调用所述环境识别算法模型分析所述图像数据中的用户周边环境场景是否为需静音场景。
所述环境识别算法模型用于：调用所述AI算法库中对应的算子分析所述图像数据中的用户周边环境场景是否为需静音场景，并在确定所述图像数据由室外场景数据变化为需静音场景场景数据时，将环境场景识别结果上报至所述AI应用层。
所述AI应用层用于：根据所述环境场景识别结果，向所述主控制器上报所述AI消息。
可选地，在一些实施例中，所述环境识别算法模型为神经网络算法模型，所述神经网络算法模型为根据大规模的目标环境场景数据进行监督学习训练而成的。
可选地，在一些实施例中，所述AI算法库固化在所述协处理器的硬件中。
可选地，在一些实施例中，所述协处理器520还包括：硬件加速器模块，用于对所述环境识别算法模型调用AI算法库模块，并分析所述图像数据中的用户周边环境场景是否为需静音场景的过程进行加速。
可选地，在一些实施例中，在所述主控制器510根据接收到的所述AI消息开启相应的终端工作模式之前，所述主控制器510处于休眠状态。
本申请实施例还提供了一种计算机可读存储介质，包括计算机程序，当该计算机程序在终端上运行时，使得该终端执行如步骤110-140等中所述的方法。
本申请实施例还提供了一种计算机程序产品，当该计算机程序产品在终端上运行时，使得该终端执行如步骤步骤110-140等中所述的方法。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请实施例的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
另外，本申请实施例的各个方面或特征可以实现成方法、装置或使用标准编程和/或工程技术的制品。本申请实施例中使用的术语“制品”涵盖可从任何计算机可读器件、载体或介质访问的计算机程序。例如，计算机可读介质可以包括，但不限于:磁存储器件(例如，硬盘、软盘或磁带等)，光盘(例如，压缩盘(compact disc，CD)、数字通用盘(digital versatile disc，DVD)等)，智能卡和闪存器件(例如，可擦写可编程只读存储器(erasable programmable read-only memory，EPROM)、卡、棒或钥匙驱动器等)。另外，本文描述的各种存储介质可代表用于存储信息的一个或多个设备和/或其它机器可读介质。术语“机器可读介质”可包括但不限于，无线信道和能够存储、包含和/或承载指令和/或数据的各种其它介质。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组 件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请实施例的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (28)

  1. 一种用于识别环境场景的方法，其特征在于，包括：
    终端通过低功耗摄像头实时获取图像数据，所述低功耗摄像头一直开启；
    所述终端根据所述图像数据，分析用户周边环境场景是否为目标环境场景；
    所述终端确定所述用户周边环境场景变化为所述目标环境场景；
    所述终端开启人工智能AI对应的工作模式。
  2. 根据权利要求1所述的方法，其特征在于，所述终端根据所述图像数据，分析用户周边环境场景是否为目标环境场景，包括：
    所述终端根据所述图像数据，调用环境识别算法模型分析用户周边环境场景是否为目标环境场景。
  3. 根据权利要求1或2所述的方法，其特征在于，所述目标环境场景为需静音场景。
  4. 根据权利要求3所述的方法，其特征在于，所述终端开启人工智能AI对应的工作模式，包括：
    所述终端确定所述用户周边环境场景变化为所述需静音场景；
    所述终端将工作模式调整为震动或静音的工作模式。
  5. 根据权利要求2至4中任一项所述的方法，其特征在于，所述环境识别算法模型为神经网络算法模型，所述神经网络算法模型为根据大规模的目标环境场景数据进行监督学习训练而成的，
    所述终端根据所述图像数据，调用环境识别算法模型分析用户周边环境场景是否为目标环境场景，包括：
    所述终端将所述图像数据输入到所述神经网络算法模型中，所述神经网络算法模型调用AI算子库中对应的算子分析所述图像数据中的用户周边环境数据是否为目标环境数据。
  6. 根据权利要求5所述的方法，其特征在于，所述AI算子库固化在所述终端的硬件中。
  7. 根据权利要求5或6所述的方法，其特征在于，所述终端将所述图像数据输入到所述神经网络算法模型中，所述神经网络算法模型调用人工智能AI算子库中对应的算子分析所述图像数据中的用户周边环境数据是否为目标环境数据，包括：
    所述神经网络算法模型通过硬件加速器调用所述AI算子库中对应的算子，并分析所述图像数据中的用户周边环境数据是否为目标环境数据。
  8. 根据权利要求1至7中任一项所述的方法，其特征在于，在所述终端开启人工智能AI对应的工作模式之前，所述终端处于休眠状态。
  9. 一种用于识别环境场景的芯片，其特征在于，包括：协处理器、主处理器，所述协处理器与所述主处理器相连，
    所述协处理器用于执行以下操作：
    通过低功耗摄像头实时获取图像数据，所述低功耗摄像头与所述协处理器相连，所述低功耗摄像头一直开启；
    根据所述图像数据分析用户周边环境场景是否为目标环境场景；
    确定所述用户周边环境场景变化为所述目标环境场景，向所述主处理器发送AI消息；
    所述主处理器用于：根据接收到的所述AI消息开启所述AI对应的工作模式。
  10. 根据权利要求9所述的芯片，其特征在于，所述协处理器具体用于：
    根据所述图像数据，调用环境识别算法模型分析用户周边环境场景是否为目标环境场景。
  11. 根据权利要求9或10所述的芯片，其特征在于，所述目标环境场景为需静音场景。
  12. 根据权利要求11所述的芯片，其特征在于，所述协处理器包括：AI引擎模块、环境识别算法模型、AI算法库模块，AI应用层模块，
    所述AI引擎模块用于：根据所述低功耗摄像头采集的图像数据，调用所述环境识别算法模型分析所述图像数据中的用户周边环境场景是否为需静音场景；
    所述环境识别算法模型用于：调用所述AI算法库中对应的算子分析所述图像数据中的用户周边环境场景是否为需静音场景，并在确定所述图像数据由室外场景数据变化为需静音场景数据时，将环境场景识别结果上报至所述AI应用层；
    所述AI应用层用于：根据所述环境场景识别结果，向所述主控制器上报所述AI消息。
  13. 根据权力要求12所述的芯片，其特征在于，所述主处理器具体用于：
    根据接收到的所述AI消息，将工作模式调整为震动或静音的工作模式。
  14. 根据权利要求9至13中任一项所述的芯片，其特征在于，所述环境识别算法模型为神经网络算法模型，所述神经网络算法模型为根据大规模的目标环境场景数据进行监督学习训练而成的。
  15. 根据权利要求12至14中任一项所述的芯片，其特征在于，所述AI算子库固化在所述协处理器的硬件中。
  16. 根据权利要求12至15中任一项所述的芯片，其特征在于，所述协处理器还包括：
    硬件加速器模块，用于对所述环境识别算法模型调用AI算法库模块，并分析所述图像数据中的用户周边环境场景是否为需静音场景的过程进行加速。
  17. 根据权利要求9至16中任一项所述的芯片，其特征在于，在所述主控制器根据接收到的所述AI消息开启相应的终端工作模式之前，所述主控制器处于休眠状态。
  18. 一种终端，其特征在于，包括：协处理器、主处理器、低功耗摄像头，所述协处理器与所述主处理器相连，所述低功耗摄像头与所述协处理器相连，
    所述协处理器用于执行以下操作：
    通过低功耗摄像头实时获取图像数据，所述低功耗摄像头与所述协处理器相连，所述低功耗摄像头一直开启；
    根据所述图像数据分析用户周边环境场景是否为目标环境场景；
    确定所述用户周边环境场景变化为所述目标环境场景，向所述主处理器发送AI消息；
    所述主处理器用于：根据接收到的所述AI消息开启所述AI对应的工作模式。
  19. 根据权利要求18所述的终端，其特征在于，所述协处理器具体用于：
    根据所述图像数据，调用环境识别算法模型分析用户周边环境场景是否为目标环境场景。
  20. 根据权利要求18或19所述的终端，其特征在于，所述目标环境场景为需静音场景。
  21. 根据权利要求18至20中任一项所述的终端，其特征在于，所述协处理器包括：AI引擎模块、环境识别算法模型、AI算法库模块，AI应用层模块，
    所述AI引擎模块用于：根据所述低功耗摄像头采集的图像数据，调用所述环境识别算法模型分析所述图像数据中的用户周边环境场景是否为需静音场景；
    所述环境识别算法模型用于：调用所述AI算法库中对应的算子分析所述图像数据中的用户周边环境场景是否为需静音场景，并在确定所述图像数据由室外场景数据变化为需静音场景数据时，将环境场景识别结果上报至所述AI应用层；
    所述AI应用层用于：根据所述环境场景识别结果，向所述主控制器上报所述AI消息。
  22. 根据权利要求21所述的终端，其特征在于，所述主处理器具体用于：
    接收到的所述AI消息，将工作模式调整为震动或静音的工作模式。
  23. 根据权利要求18至22中任一项所述的终端，其特征在于，所述环境识别算法模型为神经网络算法模型，所述神经网络算法模型为根据大规模的目标环境场景数据进行监督学习训练而成的，
    所述协处理器具体用于：
    将所述图像数据输入到所述神经网络算法模型中，所述神经网络算法模型调用AI算子库中对应的算子分析所述所述图像数据中的用户周边环境数据是否为目标环境数据。
  24. 根据权利要求23所述的终端，其特征在于，所述AI算子库固化在所述协处理器的硬件中。
  25. 根据权利要求21至24中任一项所述的终端，其特征在于，所述协处理器还包括：
    硬件加速器模块，用于对所述环境识别算法模型调用AI算法库模块，并分析所述图像数据中的用户周边环境场景是否为需静音场景的过程进行加速。
  26. 根据权利要求18至25中任一项所述的终端，其特征在于，在所述主控制器根据接收到的所述AI消息开启相应的终端工作模式之前，所述主控制器处于休眠状态。
  27. 一种计算机存储介质，其特征在于，包括计算机程序，当该计算机程序在所述终端上运行时，使得该终端执行如权利要求1至8中任一项所述的方法。
  28. 一种计算机程序产品，其特征在于，包括计算机程序，当该计算机程序在所述终端上运行时，使得该终端执行如权利要求1至8中任一项所述的方法。
PCT/CN2018/110390 2018-10-16 2018-10-16 一种识别环境场景的方法、芯片、终端 WO2020077522A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2018/110390 WO2020077522A1 (zh) 2018-10-16 2018-10-16 一种识别环境场景的方法、芯片、终端
CN201880091730.3A CN111903113A (zh) 2018-10-16 2018-10-16 一种识别环境场景的方法、芯片、终端

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/110390 WO2020077522A1 (zh) 2018-10-16 2018-10-16 一种识别环境场景的方法、芯片、终端

Publications (1)

Publication Number Publication Date
WO2020077522A1 true WO2020077522A1 (zh) 2020-04-23

Family

ID=70282870

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/110390 WO2020077522A1 (zh) 2018-10-16 2018-10-16 一种识别环境场景的方法、芯片、终端

Country Status (2)

Country Link
CN (1) CN111903113A (zh)
WO (1) WO2020077522A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114245078A (zh) * 2021-12-17 2022-03-25 广州西麦科技股份有限公司 应用多种Al识别算法对现场作业安全管控的方法及装置
CN117130287A (zh) * 2023-09-05 2023-11-28 广东保伦电子股份有限公司 一种基于光学检测和ai算法的智能控制系统及方法

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114067243A (zh) * 2021-11-10 2022-02-18 清华大学 自动驾驶场景识别方法、系统、设备及存储介质
CN114968387A (zh) * 2022-06-07 2022-08-30 三星电子（中国）研发中心 外接设备切换方法和装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106303077A (zh) * 2016-09-18 2017-01-04 珠海市魅族科技有限公司 移动终端工作模式设置方法及装置
CN106406706A (zh) * 2016-09-23 2017-02-15 乐视控股（北京）有限公司 基于终端的控制方法及装置
CN106534534A (zh) * 2016-11-09 2017-03-22 广东欧珀移动通信有限公司 工作模式的切换方法及移动终端
CN107861623A (zh) * 2017-11-30 2018-03-30 珠海市魅族科技有限公司 一种终端控制方法、装置、终端及可读存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101902531A (zh) * 2010-08-10 2010-12-01 深圳市同洲电子股份有限公司 切换情景模式的方法、装置和移动终端
US20120212636A1 (en) * 2011-02-23 2012-08-23 Canon Kabushiki Kaisha Image capture and post-capture processing
CN106850940A (zh) * 2016-11-29 2017-06-13 维沃移动通信有限公司 一种状态的切换方法及移动终端
CN108401064B (zh) * 2017-02-07 2021-08-31 中兴通讯股份有限公司 一种终端工作控制方法、装置及终端
CN107231476A (zh) * 2017-05-31 2017-10-03 深圳市邦华电子有限公司 移动终端及其情景模式设置方法、装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106303077A (zh) * 2016-09-18 2017-01-04 珠海市魅族科技有限公司 移动终端工作模式设置方法及装置
CN106406706A (zh) * 2016-09-23 2017-02-15 乐视控股（北京）有限公司 基于终端的控制方法及装置
CN106534534A (zh) * 2016-11-09 2017-03-22 广东欧珀移动通信有限公司 工作模式的切换方法及移动终端
CN107861623A (zh) * 2017-11-30 2018-03-30 珠海市魅族科技有限公司 一种终端控制方法、装置、终端及可读存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"Promote AI to the terminal and expand the frontier application scenarios of the Snapdragon AI chip", 6 August 2018 (2018-08-06), pages 1 - 4, XP055703590, Retrieved from the Internet <URL:https://news.mydrivers.com/1/589/589583.htm> *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114245078A (zh) * 2021-12-17 2022-03-25 广州西麦科技股份有限公司 应用多种Al识别算法对现场作业安全管控的方法及装置
CN114245078B (zh) * 2021-12-17 2024-04-09 广州西麦科技股份有限公司 应用多种Al识别算法对现场作业安全管控的方法及装置
CN117130287A (zh) * 2023-09-05 2023-11-28 广东保伦电子股份有限公司 一种基于光学检测和ai算法的智能控制系统及方法
CN117130287B (zh) * 2023-09-05 2024-06-25 广东保伦电子股份有限公司 一种基于光学检测和ai算法的智能控制系统及方法

Also Published As

Publication number Publication date
CN111903113A (zh) 2020-11-06

Similar Documents

Publication Publication Date Title
EP3872807B1 (en) Voice control method and electronic device
WO2020077523A1 (zh) 一种用于识别用户行为的方法、芯片、终端
WO2020077522A1 (zh) 一种识别环境场景的方法、芯片、终端
EP4030422B1 (en) Voice interaction method and device
US20220310095A1 (en) Speech Detection Method, Prediction Model Training Method, Apparatus, Device, and Medium
EP3923274A1 (en) Voice interaction method and electronic device
WO2020077540A1 (zh) 一种信息处理方法及电子设备
CN109949795A (zh) 一种控制智能设备交互的方法及装置
CN107172307A (zh) 闹钟响铃控制方法、装置及存储介质
US11720814B2 (en) Method and system for classifying time-series data
WO2021057537A1 (zh) 一种卡顿预测的方法、数据处理的方法以及相关装置
CN111343346B (zh) 基于人机对话的来电代接方法、装置、存储介质及设备
CN109389220A (zh) 神经网络模型的处理方法、装置、电子设备以及存储介质
CN107464253A (zh) 眉毛定位方法及装置
CN105872976A (zh) 定位方法和装置
CN109034747A (zh) 任务提醒方法及装置
CN106021072A (zh) 信息显示方法及装置
CN115113751A (zh) 调整触摸手势的识别参数的数值范围的方法和装置
US20210004702A1 (en) System and method for generating information for interaction with a user
WO2022188551A1 (zh) 信息处理方法与装置、主控设备和受控设备
CN110515623A (zh) 图形操作实现方法、装置、电子设备及存储介质
CN105450850A (zh) 用于提供信息的方法及其电子设备
US20240094972A1 (en) Page Display Method and Apparatus, Electronic Device, and Readable Storage Medium
CN114765026A (zh) 一种语音控制方法、装置及系统
CN114764300B (zh) 一种窗口页面的交互方法、装置、电子设备以及可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18937077

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18937077

Country of ref document: EP

Kind code of ref document: A1