WO2017173976A1 - Procédé permettant de réaliser une commande multidimensionnelle, terminal intelligent et dispositif de commande - Google Patents

Procédé permettant de réaliser une commande multidimensionnelle, terminal intelligent et dispositif de commande Download PDF

Info

Publication number
WO2017173976A1
WO2017173976A1 PCT/CN2017/079444 CN2017079444W WO2017173976A1 WO 2017173976 A1 WO2017173976 A1 WO 2017173976A1 CN 2017079444 W CN2017079444 W CN 2017079444W WO 2017173976 A1 WO2017173976 A1 WO 2017173976A1
Authority
WO
WIPO (PCT)
Prior art keywords
controller
control
motion estimation
scene information
smart terminal
Prior art date
Application number
PCT/CN2017/079444
Other languages
English (en)
Chinese (zh)
Inventor
赵秋林
黄宇轩
刘成刚
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2017173976A1 publication Critical patent/WO2017173976A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Definitions

  • the present application relates to, but is not limited to, intelligent technology, and more particularly to a method, an intelligent terminal and a controller for implementing multi-dimensional control.
  • this multi-dimensional user experience can only be experienced on a dedicated movie.
  • the control commands of the multi-dimensional experience are synchronized with the movie in advance, for example, at the corresponding show time point, the corresponding controller is issued.
  • Control commands to allow the controller to control effects such as vibrations, blows, smoke, bubbles, smells, scenery, and character performances. That is to say, the realization of this new entertainment effect is currently limited in the use of the family.
  • the present application provides a method, an intelligent terminal and a controller for realizing multi-dimensional control, which can add multi-dimensional experience effects to the screening content in real time, and is applicable to ordinary families.
  • the application provides a method for implementing multi-dimensional control, including:
  • the smart terminal analyzes the obtained currently played video content to identify the scene information corresponding to the video content
  • the smart terminal sends the scene information to the controller, so that the controller starts multi-dimensional control according to the scene information.
  • the analyzing the acquired video content to identify the The scenario information corresponding to the video content may include:
  • the video frame is sampled and analyzed, and the candidate object is searched, wherein for each sample frame, a motion estimation vector is acquired, and a plurality of regions in the macroblock set with a large motion estimation vector are defined as a marker region;
  • the smart terminal continuously detects key frames in the currently played video frame. If there is a marked area in a sequence of video frames that are long in a preset period, the smart terminal starts sampling and analyzing the video frame. A key frame in the sequence identifies, for each sample frame, a candidate object within the video frame and a location thereof to identify the scene information.
  • the delineating a plurality of regions in the macroblock set having a large motion estimation vector as the marker region may include:
  • the obtained motion estimation vector is divided into the following two categories by using a classification algorithm: a macroblock with a large motion estimation vector and a macroblock with a small motion estimation vector;
  • a plurality of regions in the macroblock set having a large motion estimation vector are defined as marker regions; objects located outside the marker regions are used as reference objects.
  • the present application further provides a method for implementing multi-dimensional control, comprising: the controller identifying an instruction that needs to start multi-dimensional experience control according to the obtained scene information corresponding to the currently played video content, and performing corresponding control.
  • a correspondence between different object categories and control information is preset in the controller
  • the determining, according to the obtained scenario information, the instruction that the multi-dimensional experience control needs to be started may include: determining that the object in the obtained scene information belongs to a preset object type of the trigger control, and determining that the preset trigger condition is met, determining The instruction of the multidimensional experience control is initiated.
  • the controller may include at least one of the following: a vibration controller, an odor controller, a spray controller, a light controller, and a sound controller.
  • distributed deployment or centralized deployment, is employed between multiple controllers.
  • the application further provides a method for implementing a multi-dimensional experience, including:
  • the smart terminal analyzes the obtained currently played video content to identify and initiate the request.
  • the smart terminal determines, according to the identified scenario information, whether to start multi-dimensional experience control
  • the corresponding control information is sent to the corresponding controller.
  • the foregoing method may further include:
  • the smart terminal listens to a query command from one or more controllers, and returns its own device description information to the controller that initiates the query request;
  • the analyzing the obtained video content, and identifying the scenario information corresponding to the controller that initiated the request may include:
  • the video frame is sampled and analyzed, and the candidate object is searched, wherein for each sample frame, a motion estimation vector is acquired, and a plurality of regions in the macroblock set with a large motion estimation vector are defined as a marker region;
  • Each sample frame identifies a candidate object associated with a controller that initiates the query and establishes a session within the video frame and a location thereof to identify the scene information corresponding to the controller that initiated the query and establishes the session.
  • the delineating a plurality of regions in the macroblock set having a large motion estimation vector as the marker region may include:
  • the obtained motion estimation vector is divided into the following two categories by using a classification algorithm: a macroblock with a large motion estimation vector and a macroblock with a small motion estimation vector;
  • a plurality of regions in the macroblock set having a large motion estimation vector are defined as marker regions; objects located outside the marker regions are used as reference objects.
  • a correspondence between different object categories and control information is preset in the smart terminal
  • Determining whether the multi-dimensional experience control needs to be started according to the identified scene information includes: the object in the obtained scene information belongs to a preset object type of trigger control, and when the preset trigger condition is met, the corresponding multi-dimensional is started. Experience control.
  • the present application further provides an intelligent terminal, including: a first analysis module and a broadcast module; wherein
  • a first analysis module configured to analyze the obtained currently played video content, to identify scene information corresponding to the video content
  • the broadcast module is configured to send the identified scene information to the controller, so that the controller starts the multi-dimensional control according to the scene information.
  • the first analysis module may be configured to: when playing a video, sample and analyze a video frame, obtain a motion estimation vector for each sample frame; and use a classification algorithm to divide the obtained motion estimation vector into the following: Two types: a macroblock with a large motion estimation vector and a macroblock with a small motion estimation vector; and a plurality of regions in a macroblock set with a large motion estimation vector are defined as a marker region;
  • the application further provides an intelligent terminal, comprising: a second analysis module and a determining module; wherein
  • the second analysis module is configured to analyze the obtained currently played video content to identify the scenario information corresponding to the controller that initiated the request;
  • the determining module is configured to determine whether the multi-dimensional experience control needs to be started according to the identified scenario information. When it is determined that the multi-dimensional experience control needs to be started, the corresponding control information is sent to the corresponding controller.
  • the smart terminal may further include: an establishing module configured to listen to a query command from one or more controllers, and return device description information of the smart terminal to which the smart terminal belongs to the controller that initiates the query request; Establish a session with the controller that initiated the session.
  • the second analysis module may be configured to:
  • the motion estimation vector is obtained for each sample frame.
  • the motion estimation vectors obtained by the classification algorithm are classified into the following two types: macroblocks with large motion estimation vectors and macros with small motion estimation vectors. Block; delineating several regions in a macroblock set with a large motion estimation vector Marked area; an object located outside the marked area is called a reference;
  • the determining module may be configured to: when the object in the obtained scene information belongs to a preset trigger-controlled object according to a correspondence between different object categories and control information set in advance When the category is met and the preset trigger condition is met, the corresponding multi-dimensional experience control is started, and the corresponding control information is sent to the corresponding controller.
  • the application further provides a controller, comprising: an acquisition module and a control module; wherein
  • Obtaining a module configured to obtain scene information corresponding to the currently played video content
  • the control module is configured to perform corresponding control when it is determined according to the obtained scenario information that the multi-dimensional experience control needs to be started.
  • a correspondence between different object categories and control information is preset in the control module
  • the control module may be configured to start the multi-dimensional experience control when the object in the obtained scene information belongs to a preset object type of trigger control and meets a preset trigger condition.
  • the obtaining module is further configured to: send a query command to query device description information of the smart terminal in the current network, and listen to information broadcast by the smart terminal.
  • the technical solution of the present application includes the smart terminal analyzing the obtained currently played video content to identify the scene information corresponding to the video content; the intelligent terminal sends the scene information to the controller, so that the controller starts multi-dimensional control according to the scene information. Or, after the multi-dimensional experience function is started, the smart terminal analyzes the currently played video content to obtain scenario information corresponding to the controller that initiates the request; the smart terminal determines, according to the obtained scenario information, whether to start multi-dimensional experience control; When the multi-dimensional experience control needs to be started, the corresponding control information is sent to the corresponding controller.
  • the technical solution provided by the present application utilizes an intelligent terminal to implement audio and video detection, to identify a current video playing scene, and control various controllers according to the identified various scenes to reconstruct the current playing.
  • the scene enables real-time multi-dimensional experience effects on the content of the show, and is suitable for ordinary families.
  • FIG. 1 is a flowchart of a method for implementing a multi-dimensional experience according to an embodiment of the present invention
  • FIG. 2 is a flowchart of another method for implementing a multi-dimensional experience according to an embodiment of the present invention
  • FIG. 3 is a schematic structural diagram of a smart terminal according to an embodiment of the present invention.
  • FIG. 4 is a schematic structural diagram of another smart terminal according to an embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of a controller according to an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of a networking architecture of a controller deployed in a centralized manner according to an embodiment of the present invention
  • FIG. 7 is a schematic diagram of a networking architecture of a controller deployed in a distributed manner according to an embodiment of the present invention.
  • FIG. 1 is a flowchart of a method for implementing multi-dimensional control according to an embodiment of the present invention. As shown in FIG. 1 , the method includes:
  • Step 100 The smart terminal analyzes the obtained currently played video content to identify the scene information corresponding to the video content.
  • the smart terminal After starting the multi-dimensional experience function, first, when the smart terminal plays the video, sample and analyze the video frame, and try to search for candidate objects, such as flowers (such as corresponding wind), grass, rock slurry (such as corresponding vibration), etc. Frame, obtain motion estimation vector; use classification algorithm such as k-means clustering
  • the analysis divides the obtained motion estimation vectors into two categories: macroblocks with large motion estimation vectors and macroblocks with small motion estimation vectors. A plurality of regions in a macroblock set in which the motion estimation vector is large are defined as a marker region. If a marked area is too small, discard the marked area. An object located outside the marked area serves as a reference for the large background. In this way, the possible areas where key candidate objects exist are found.
  • a preset area such as a rectangular area
  • a preset threshold such as 80% (adjustable)
  • the area is considered to be a marker. region.
  • the area of the marked marked area occupies a ratio of the total area smaller than the preset area, such as 10% (adjustable)
  • the marked area is discarded.
  • the smart terminal continuously detects the key frame in the obtained video frame, that is, the I frame. If there is always a marked area in a sequence of video frames that are long in a preset period, the smart terminal starts sampling and analyzing the video.
  • the key frame in the frame sequence identifies and identifies the candidate object and the location in the video frame by using an algorithm such as a neural network for each sample frame, thereby identifying the scene information. In this way, the identification of key candidate objects is achieved.
  • the candidate object identified in the marked area of the video frame sequence is marked as the candidate object category if the following conditions are met: 1) The object class exists in the marked area of successive video frame sequences; 2) each object of the object class continues to change relative to the reference vector of each video sequence.
  • the scene information further includes: recording additional parameters such as an object duration, an object position moving relative speed, a number, and the like.
  • the neural network used above can adopt the structure of AlexNet: a total of 8 layers, the first 5 layers are convolution layers, and the last 3 layers are fully connected layers. Among them, the last layer uses the softmax classifier. Among them, in the first five layers of the convolutional layer, the first layer is a convolutional layer, which is convoluted using a specific template interval, and then uses ReLU as an activation function, and is polled after regularization, and the obtained result is used as a second layer convolution.
  • Layer input the following four layers of convolutional layer and the first layer are similar, but the convolution template with lower dimension is used; in the last three layers of the full connection layer, after the last three layers of ReLU, the dropout is fully connected; finally, softmax is used. As a lost function.
  • the edge contour of the flower For example, if you use a neural network to detect a large area of flowers in the current picture, you can find the edge contour of the flower. If you also detect that the flower has a large amplitude of shaking to the right, then swing according to the flower. The direction can be inferred that the wind blows from left to right, and the level of the wind can be derived according to the amplitude of the flower swing; if a character is also detected in the picture, the position and number of the character are marked, and The frame finds the speed of relative movement between characters and the like. The information obtained is the scene information required in this step.
  • Step 101 The smart terminal sends the identified scene information to the controller, so that the controller starts multi-dimensional control according to the scene information.
  • the smart terminal sends the identified scene information to the controller, such as broadcasting the identified scene information.
  • the scene information may include: the type of flower, the approximate number of flowers; the direction of wind blowing and the level of wind; the number of characters and the speed of relative movement.
  • the control information is used for the controller that needs to start the multi-dimensional experience control to perform corresponding control.
  • the controller further includes: the controller identifies, according to the obtained scene information corresponding to the currently played video content, an instruction that needs to start multi-dimensional experience control, and performs corresponding control.
  • the controller in the present application may include, but is not limited to, at least one of the following: a vibration controller, an odor controller, a spray controller, a light controller, and a sound controller.
  • controllers can be distributed or centralized. When distributed deployment is used, each controller communicates with the intelligent terminal; when centralized deployment, multiple controllers can be placed in one device, such as a wearable device, which is more convenient for the user. Experience. Among them, the controller and the intelligent terminal can communicate by means of Ethernet, WiFi, Bluetooth, and the like.
  • a correspondence relationship between different object categories and control information is set in advance, and when the object in the obtained scene information belongs to a preset object type of the trigger control, and the preset trigger condition is satisfied, , determine the instruction to start the corresponding multi-dimensional experience control.
  • the correspondence may be set as: when the obtained object information belongs to an object category that triggers vibration, such as rock, and meets the trigger condition, such as the number of objects is greater than one and the speed is greater than 1/8 of the screen. Seconds, lasting more than 3 seconds, the vibration controller is activated to trigger the vibration effect;
  • the correspondence may be set to: when the object in the obtained scene information belongs to an object category that triggers the generation of odor, such as osmanthus, and meets the trigger condition such as the continuous appearance time > 6 seconds, and the number > 10 Then, the odor controller is activated to trigger the scent of osmanthus fragrance.
  • the corresponding relationship may be: when the obtained object information belongs to the object category that triggers the generated sound, if a task appears in the screen, and the trigger condition is met, such as the position, moving direction, and movement of the character. At the speed, etc., the sound controller is activated to trigger a gradual process in which the footstep moves in accordance with the direction in which the character moves.
  • FIG. 2 is a flowchart of another method for implementing multi-dimensional control according to an embodiment of the present invention. As shown in FIG. 2, the method includes:
  • Step 200 The smart terminal analyzes the obtained currently played video content to identify the scenario information corresponding to the controller that initiated the request.
  • the method may further include: after the controller is started, sending a query command to the smart terminal to query the device information of the smart terminal in the current network, and listening to the information broadcast by the smart terminal;
  • the intelligent terminal acts as a convergence point to listen to the query from the controller, and when the query is queried, returns its own device description information to the controller that initiates the query request;
  • the controller receiving the query response acts as a client to initiate a session to the smart terminal and establishes a session between the smart terminal and the controller.
  • this step the smart terminal collects corresponding scenario information for the request of the controller.
  • the vibration controller is used to initiate the query request.
  • the smart terminal only recognizes the object category such as rock that triggers the vibration, that is, the object in the scene information returned at this time only has the object category that triggers the vibration.
  • Step 201 The smart terminal determines, according to the identified scenario information, whether to start multi-dimensional experience control.
  • a correspondence relationship between different object categories and control information is set in advance, and the object in the obtained scene information belongs to a preset object type of the trigger control, and the preset trigger condition is satisfied.
  • the corresponding multidimensional experience control is started.
  • step 102 The specific implementation of this step is consistent with step 102, and details are not described herein again.
  • Step 202 When it is determined that the multi-dimensional experience control needs to be started, the corresponding control information is sent to the corresponding controller.
  • the intelligent terminal directly delivers the final control information to the controller, and the controller only needs to start and trigger the corresponding action according to the received control command.
  • FIG. 3 is a schematic structural diagram of a smart terminal according to an embodiment of the present invention. As shown in FIG. 3, the method includes at least a first analysis module 300 and a broadcast module 301.
  • the first analysis module 300 is configured to analyze the acquired currently played video content to identify scene information corresponding to the video content.
  • the broadcast module 301 is configured to send the identified scene information to the controller, so that the controller starts the multi-dimensional control according to the scene information.
  • the first analysis module 300 can be configured to:
  • the motion estimation vector is obtained for each sampling frame;
  • the motion estimation vectors obtained by the classification algorithm such as k-means cluster analysis are divided into the following two categories: motion A macroblock having a large vector and a macroblock having a small motion estimation vector are estimated; a plurality of regions in the macroblock set having a large motion estimation vector are defined as a marked region; if a marked region is too small, the marked region is discarded. An object located outside the marked area is called a reference object;
  • Each sample frame is identified by a neural network or the like to locate a candidate object in the video frame and a location thereof, thereby obtaining scene information.
  • FIG. 4 is a schematic structural diagram of another smart terminal according to an embodiment of the present invention. As shown in FIG. 4, the method includes at least a second analysis module 401 and a determining module 402.
  • the second analysis module 401 is configured to analyze the obtained currently played video content to identify the scenario information corresponding to the controller that initiated the request;
  • the determining module 402 is configured to determine whether the multi-dimensional experience control needs to be started according to the identified scenario information. When it is determined that the multi-dimensional experience control needs to be started, the corresponding control information is sent to the corresponding controller.
  • the smart terminal shown in FIG. 4 may further include an establishing module 400 configured to listen to a query command from a certain controller or some controllers, and return device description information of the smart terminal to which the smart terminal belongs to the controller that initiates the query request; A session is established between the controllers of the session.
  • an establishing module 400 configured to listen to a query command from a certain controller or some controllers, and return device description information of the smart terminal to which the smart terminal belongs to the controller that initiates the query request; A session is established between the controllers of the session.
  • the second analysis module 401 can be configured to:
  • Each sampling frame identifies a candidate object and a location in the video frame that are related to the controller that initiated the query and establishes the session through an algorithm such as a neural network, thereby identifying scene information corresponding to the controller that initiated the query and establishes the session.
  • the determining module 402 may be configured to: when the object in the obtained scene information belongs to a preset trigger-controlled object category according to a preset relationship between different object categories and control information, and meets a preset trigger condition The corresponding multi-dimensional experience control is started, and the corresponding control information is sent to the corresponding controller.
  • FIG. 5 is a schematic structural diagram of a controller according to an embodiment of the present invention. As shown in FIG. 5, the method includes at least an obtaining module 500 and a control module 501.
  • the obtaining module 500 is configured to obtain scene information corresponding to the currently played video content.
  • the control module 501 is configured to perform corresponding control when it is determined that the multi-dimensional experience control needs to be started according to the obtained scenario information.
  • the control module 501 may be configured with a corresponding relationship between different object types and control information in advance; the control module 501 may be configured to: when the obtained object in the scene information belongs to a preset trigger-controlled object category, and meets Multi-dimensional experience control is initiated when a pre-set trigger condition is set.
  • the obtaining module 500 is further configured to: send a query command to query device description information of the smart terminal in the current network, and listen to information broadcast by the smart terminal.
  • FIG. 6 is a schematic diagram of a networking architecture of a centralized deployment of a controller according to an embodiment of the present invention.
  • a centralized deployment is adopted between multiple controllers, such as Wear the device.
  • the vibration controller (such as the vibration embedded in the smart pants)
  • the controller initiates the query request as an example, and in the first embodiment, the smart terminal determines whether the vibration controller needs to be activated to trigger the vibration effect.
  • This embodiment may include:
  • an inquiry command is sent to the intelligent terminal to query the device description information of the intelligent terminal in the current network, and listen to the broadcast information of the intelligent terminal; the intelligent terminal acts as a convergence point, and when the monitoring device has a vibration controller to initiate an inquiry, Reading its own device description information and returning it to the vibration controller through the query response; the vibration controller acts as a client to initiate a session request, and the intelligent terminal receives the session request and establishes a session between itself and the vibration controller.
  • the video frame is sampled and analyzed, and the candidate object is searched for, that is, the motion estimation vector is acquired for each sample frame.
  • the obtained motion estimation vectors of the video frames are classified into the following two types by using a classification algorithm: macroblocks with large motion estimation vectors and macroblocks with small motion estimation vectors.
  • a plurality of regions in a macroblock set in which the motion estimation vector is large are defined as a marker region. If a marked area is too small, discard the marked area. An object located outside the marked area is called a reference.
  • the frames in the sequence of video frames are sampled and analyzed, and each sample frame is identified by a neural network or the like to locate the main object and the location in the video frame.
  • the neural network can adopt the structure of AlexNet: a total of 8 layers, the first 5 layers are convolution layers, and the last 3 layers are fully connected layers. Among them, the last layer uses the softmax classifier.
  • the first layer is a convolutional layer, which is convoluted using a specific template interval, and then uses ReLU as an activation function, and is polled after regularization, and the obtained result is used as a second layer convolution.
  • Layer input the following four layers of convolutional layer and the first layer are similar, but the convolution template with lower dimension is used; in the last three layers of the full connection layer, after the last three layers of ReLU, the dropout is fully connected; finally, softmax is used. As a lost function.
  • the candidate object identified in the marked area of the video frame sequence is marked as the candidate object category if the following conditions are met: 1) The object class exists in the marked area of successive video frame sequences; 2) each object of the object class continues to change relative to the reference vector of each video sequence.
  • the scene information further includes: recording additional parameters such as an object duration, an object position moving relative speed, a number, and the like.
  • the vibration controller there is a correspondence between different object categories and control information in the smart terminal, when the object in the obtained scene information belongs to a preset object type of trigger control, and the preset trigger condition is satisfied. , start the corresponding multi-dimensional experience control.
  • a plurality of triggering vibration correspondences are preset for the vibration controller: each trigger item is provided with a triggering object category, and a trigger condition, and the vibration effect is triggered when the trigger item is satisfied.
  • the correspondence may be set as: when the obtained object information belongs to an object category that triggers vibration, such as rock, and meets the trigger condition, such as the number of objects is greater than one and the speed is greater than 1/8 of the screen. In seconds, lasting more than 3 seconds, the vibration controller is activated to trigger the vibration effect.
  • the smart terminal only needs to send the corresponding control information, that is, the triggering vibration effect, to the vibration controller.
  • the smart terminal determines whether it is necessary to activate the odor controller to emit an odor effect, and then generates control information and sends it to the odor controller.
  • This embodiment may include:
  • the query command is sent to the smart terminal to query the device description information of the smart terminal in the current network, and listen to the broadcast information of the smart terminal; the smart terminal acts as a convergence point, and when the odor controller is inquired to initiate the query Reading its own device description information and returning it to the odor controller through the query response; the scent controller initiates a session request as a client, and the smart terminal receives the session request and establishes a session between itself and the odor controller.
  • the smart terminal classifies according to objects in the scene, and in some scenarios, it is necessary to manufacture certain environmental odors to enrich the user experience, and accordingly, identifiable objects and corresponding odors are preset.
  • the object in the obtained scene information belongs to the preset object type of the trigger control, and the preset trigger condition is met, , start the corresponding multi-dimensional experience control.
  • a plurality of triggering scent correspondences are preset for the scent controller: the object type of the trigger is specified in each trigger item, and the trigger condition, and the scent effect is triggered when the trigger item is satisfied.
  • the odor controller is activated to trigger the scent of osmanthus fragrance:
  • the smart terminal only needs to send the corresponding control information, that is, the scent of osmanthus fragrance to the odor controller.
  • FIG. 7 is a schematic diagram of a networking architecture of a controller deployed in a distributed manner according to an embodiment of the present invention. As shown in FIG. 7, in the third embodiment, a distributed deployment between multiple controllers is assumed. In the third embodiment, the smart terminal only needs to identify the set object category and broadcast the identified scene information; and each controller performs scene information belonging to its own control range whether to start the controller to trigger the multi-dimensional effect. Make a decision.
  • This embodiment may include:
  • the key frames in the currently played video frame are continuously detected.
  • the neural network detects that there is a large area of flower sea in the current picture, and after finding the edge contour of the flower, if it is detected that the flower is larger to the right
  • the amplitude of the sway then, according to the direction of the flower swing, it can be inferred that the wind blows from left to right, and the level of the wind can be derived according to the amplitude of the flower swing; if a person is also detected in the picture, then the mark is The position and number of characters, and the speed at which relative movement between characters is found through multiple frames.
  • the information obtained is the scene information.
  • the smart terminal broadcasts the obtained scene information, that is, the type of flower, the approximate number of flowers, the direction of wind blowing and the level of wind; the number of characters and the speed of relative movement.
  • the blowing controller For each blowing controller, according to the obtained scene information, the location of the location, and the corresponding relationship between the different scene information and the control information, it is determined whether the blowing is required, and the magnitude of the wind. For example: the scene information stroke is blown from left to right. If the position of the blow controller is on the left, then the corresponding wind in the scene information is blown; if the orientation of the blow controller is on the right, there is no need to trigger the blow.
  • the odor controller For each odor controller, according to the obtained scene information and the corresponding relationship between the preset scene information and the control information, the odor controller is triggered to release the fragrance of the flower in the corresponding scene information.
  • each sound controller For each sound controller, according to the obtained scene information, select the corresponding background sound such as wind and grass The sound of movement. And according to the moving speed and moving direction of the character in the scene information, and the corresponding relationship between the different scene information and the control information set in advance, and according to the channel corresponding to the sound controller itself, triggering the sound controller to select the strength of the footstep sound Or gradient, then superimpose the background sound and footsteps and output. Complete the sound output of this channel.
  • the embodiment of the invention further provides a computer readable storage medium storing computer executable instructions, which are implemented by the processor to implement the method for implementing multidimensional control according to any of the above embodiments.
  • computer storage medium includes volatile and nonvolatile, implemented in any method or technology for storing information, such as computer readable instructions, data structures, program modules or other data. Sex, removable and non-removable media.
  • Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridge, magnetic tape, magnetic disk storage or other magnetic storage device, or may Any other medium used to store the desired information and that can be accessed by the computer.
  • communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and can include any information delivery media. .
  • the embodiment of the present application provides a method, an intelligent terminal, and a controller for implementing multi-dimensional control, which implements audio and video detection by using an intelligent terminal, is used to identify a current video playing scene, and controls various controllers according to the identified various scenarios. To reconstruct the currently playing scene, real-time multi-dimensional experience effect on the screening content, and suitable for ordinary families.

Abstract

La présente invention porte sur un procédé permettant de réaliser une expérience multidimensionnelle, sur un terminal intelligent et sur un dispositif de commande. Le procédé comprend les étapes suivantes : un terminal intelligent analyse un contenu vidéo acquis en cours de lecture de sorte à reconnaître des informations de scène correspondant au contenu vidéo (100) ; et le terminal intelligent envoie les informations de scène à un dispositif de commande de telle sorte que le dispositif de commande active une commande multidimensionnelle en fonction des informations de scène (101). Dans le procédé, un terminal intelligent est utilisé pour réaliser une détection audio et vidéo de sorte à reconnaître une scène en cours de lecture dans une vidéo et des dispositifs de commande sont commandés de sorte à recréer une scène en cours de lecture en fonction de diverses scènes reconnues, ce qui permet de réaliser qu'un effet d'expérience multidimensionnelle est ajouté en temps réel dans un contenu qui doit être présenté, et est approprié pour des ménages ordinaires.
PCT/CN2017/079444 2016-04-05 2017-04-05 Procédé permettant de réaliser une commande multidimensionnelle, terminal intelligent et dispositif de commande WO2017173976A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610206745.2A CN105760141B (zh) 2016-04-05 2016-04-05 一种实现多维控制的方法、智能终端及控制器
CN201610206745.2 2016-04-05

Publications (1)

Publication Number Publication Date
WO2017173976A1 true WO2017173976A1 (fr) 2017-10-12

Family

ID=56333468

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/079444 WO2017173976A1 (fr) 2016-04-05 2017-04-05 Procédé permettant de réaliser une commande multidimensionnelle, terminal intelligent et dispositif de commande

Country Status (2)

Country Link
CN (1) CN105760141B (fr)
WO (1) WO2017173976A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110493090A (zh) * 2019-08-22 2019-11-22 三星电子（中国）研发中心 一种实现智能家庭影院的方法和系统
CN111031392A (zh) * 2019-12-23 2020-04-17 广州视源电子科技股份有限公司 媒体文件的播放方法、系统、装置、存储介质和处理器
EP3675504A1 (fr) * 2018-12-31 2020-07-01 Comcast Cable Communications LLC Données environnementales pour contenu multimédia

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105760141B (zh) * 2016-04-05 2023-05-09 中兴通讯股份有限公司 一种实现多维控制的方法、智能终端及控制器
CN106657975A (zh) * 2016-10-10 2017-05-10 乐视控股（北京）有限公司 视频播放方法及设备
CN108063701B (zh) * 2016-11-08 2020-12-08 华为技术有限公司 一种控制智能设备的方法及装置
CN107743205A (zh) * 2017-09-11 2018-02-27 广东欧珀移动通信有限公司 图像处理方法及装置、电子装置和计算机可读存储介质
CN110475159A (zh) * 2018-05-10 2019-11-19 中兴通讯股份有限公司 多媒体信息的传输方法及装置、终端
CN109388719A (zh) * 2018-09-30 2019-02-26 京东方科技集团股份有限公司 基于数字化作品的多维场景数据生成装置及方法
CN110245628B (zh) * 2019-06-19 2023-04-18 成都世纪光合作用科技有限公司 一种检测人员讨论场景的方法和装置
CN112040289B (zh) * 2020-09-10 2022-12-06 深圳创维－Rgb电子有限公司 视频播放控制方法、装置、视频播放设备和可读存储介质
CN114885189A (zh) * 2022-04-14 2022-08-09 深圳创维－Rgb电子有限公司 香氛开启的控制方法、装置、设备及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070217511A1 (en) * 2006-03-14 2007-09-20 Celestial Semiconductor, Ltd. Method and system for motion estimation with multiple vector candidates
CN105072483A (zh) * 2015-08-28 2015-11-18 深圳创维－Rgb电子有限公司 基于智能电视视频场景的智能家居设备互动方法及系统
CN105306982A (zh) * 2015-05-22 2016-02-03 维沃移动通信有限公司 移动终端界面图像的感官反馈方法及其移动终端
CN105760141A (zh) * 2016-04-05 2016-07-13 中兴通讯股份有限公司 一种实现多维控制的方法、智能终端及控制器

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101035279B (zh) * 2007-05-08 2010-12-15 孟智平 一种在视频资源中使用信息集的方法
CN103559713B (zh) * 2013-11-10 2017-01-11 深圳市幻实科技有限公司 一种提供增强现实的方法及终端
CN103679727A (zh) * 2013-12-16 2014-03-26 中国科学院地理科学与资源研究所 一种多维时空动态联动分析方法和装置
CN103970892B (zh) * 2014-05-23 2017-03-01 无锡清华信息科学与技术国家实验室物联网技术中心 基于智能家居设备的多维度观影系统控制方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070217511A1 (en) * 2006-03-14 2007-09-20 Celestial Semiconductor, Ltd. Method and system for motion estimation with multiple vector candidates
CN105306982A (zh) * 2015-05-22 2016-02-03 维沃移动通信有限公司 移动终端界面图像的感官反馈方法及其移动终端
CN105072483A (zh) * 2015-08-28 2015-11-18 深圳创维－Rgb电子有限公司 基于智能电视视频场景的智能家居设备互动方法及系统
CN105760141A (zh) * 2016-04-05 2016-07-13 中兴通讯股份有限公司 一种实现多维控制的方法、智能终端及控制器

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3675504A1 (fr) * 2018-12-31 2020-07-01 Comcast Cable Communications LLC Données environnementales pour contenu multimédia
CN110493090A (zh) * 2019-08-22 2019-11-22 三星电子（中国）研发中心 一种实现智能家庭影院的方法和系统
CN111031392A (zh) * 2019-12-23 2020-04-17 广州视源电子科技股份有限公司 媒体文件的播放方法、系统、装置、存储介质和处理器

Also Published As

Publication number Publication date
CN105760141A (zh) 2016-07-13
CN105760141B (zh) 2023-05-09

Similar Documents

Publication Publication Date Title
WO2017173976A1 (fr) Procédé permettant de réaliser une commande multidimensionnelle, terminal intelligent et dispositif de commande
CN109873951B (zh) 一种视频拍摄及播放的方法、装置、设备及介质
US11854547B2 (en) Network microphone device with command keyword eventing
US10586540B1 (en) Network microphone device with command keyword conditioning
US10554850B2 (en) Video ingestion and clip creation
CN106686404B (zh) 一种视频分析平台、匹配方法、精准投放广告方法及系统
CN106057205B (zh) 一种智能机器人自动语音交互方法
CN104620522B (zh) 通过所检测到的身体标记来确定用户兴趣
US11810597B2 (en) Video ingestion and clip creation
KR102197098B1 (ko) 콘텐츠 추천 방법 및 장치
EP2376207B1 (fr) Procédé et système de production de données permettant de commander un système permettant le rendu d'au moins un signal
CN104618446A (zh) 一种实现多媒体推送的方法和装置
CN109635616A (zh) 互动方法和设备
EP3675504A1 (fr) Données environnementales pour contenu multimédia
US20220148592A1 (en) Network Device Interaction by Range
CN111442464B (zh) 空调器及其控制方法
KR101924715B1 (ko) 디바이스 제어를 위한 적외선 시그널링의 자동 설정을 가능하게 하는 기법
US20200143823A1 (en) Methods and devices for obtaining an event designation based on audio data
KR20160099289A (ko) 영상의 전역 특징과 이동객체의 지역 특징을 융합한 동영상 검색 방법 및 그 시스템
CN111096078B (zh) 一种用于创建视频的光脚本的方法和系统
US20180006869A1 (en) Control method and system, and electronic apparatus thereof
US20230147768A1 (en) Adaptive learning system for localizing and mapping user and object using an artificially intelligent machine
CN111225233A (zh) 一种多维环境渲染系统及渲染方法
EP3777485B1 (fr) Système et procédés pour augmenter des commandes vocales à l'aide de systèmes d'éclairage connectés
US20190066711A1 (en) Voice filtering system and method

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17778640

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17778640

Country of ref document: EP

Kind code of ref document: A1