WO2023082655A1

WO2023082655A1 - Target control method and apparatus, and device and medium

Info

Publication number: WO2023082655A1
Application number: PCT/CN2022/101770
Authority: WO
Inventors: 袁青伟; 赵永俊; 蔺怀钰; 尹鹏; 孙立翔; 李连会; 武文杰; 牟晨涛
Original assignee: 青岛海尔空调电子有限公司; 青岛海尔空调器有限总公司; 海尔智家股份有限公司
Priority date: 2021-11-15
Filing date: 2022-06-28
Publication date: 2023-05-19
Also published as: CN114253396A

Abstract

A target control method, which relates to the technical field of human-machine interaction, with an aim to achieve convenient and quick triggering of a related controlled apparatus. The target control method comprises: acquiring first trigger information, which is within a preset region around a target, wherein the first trigger information comprises eye information and/or non-eye information of a person within the preset region (S1); on the basis of the first trigger information, determining, from among a plurality of controlled apparatuses, a controlled apparatus to be triggered (S2); acquiring second trigger information, which is within the preset region around the target, wherein the second trigger information comprises the eye information and/or the non-eye information of the person within the preset region (S3); on the basis of the second trigger information, determining whether to trigger the controlled apparatus to be triggered (S4); and on the basis of a trigger information-based determination result, triggering the controlled apparatus to be triggered (S5). By means of the method, a related controlled apparatus can be conveniently and quickly triggered, thereby improving the usage experience of a user.

Description

A target control method, device, equipment and medium

technical field

The present invention relates to the technical field of human-computer interaction, and specifically provides an object control method, device, equipment and medium.

Background technique

In daily life, in most scenarios, it is necessary to use limbs or objects to make the device produce corresponding actions by touching the device when opening, waking up, closing the device, and sleeping the device. For example, turn on the power of the device by pressing a switch with your hand. But for people with limited physical mobility, even such a simple touch operation is difficult to complete. In addition, although there are some device control methods at present, when there are multiple devices in the same area, how to achieve precise control of the device to be controlled is also a problem that needs to be solved urgently.

Contents of the invention

The present invention aims to solve the above-mentioned technical problems, that is, by providing a target control method to solve the problem of how to trigger the controlled device more conveniently and quickly for the existing physically handicapped personnel, and how to achieve precise control when facing multiple devices .

In the first aspect, the present invention provides a target control method, the target includes a plurality of controlled devices, the present invention includes the following steps,

Acquire first trigger information in a preset area around the target, where the first trigger information includes eye information and/or non-eye information of people in the preset area;

determining a controlled device to be triggered among the plurality of controlled devices based on the first trigger information;

Acquire second trigger information in a preset area around the target, where the second trigger information includes eye information and/or non-eye information of people in the preset area;

judging whether to trigger the controlled device to be triggered based on the second trigger information;

Based on the trigger information judgment result, trigger the controlled device to be triggered.

Optionally, acquiring trigger information in a preset area around the target includes acquiring eye information of the person, and/or acquiring non-eye information, wherein

The eye information includes pupil information or eye movement information,

The non-eye information includes any one or more of gesture information, body information, and voice information.

Optionally, determining a controlled device to be triggered among the plurality of controlled devices based on the first trigger information includes:

If the first trigger information is eye information, determine whether the eye information meets preset requirements, and if so, determine the controlled device to be triggered among the plurality of controlled devices based on the eye information ;

Judging whether to trigger the controlled device to be triggered based on the second trigger information includes:

Judging whether the second trigger information corresponds to the controlled device to be triggered, and if so, triggering the controlled device to be triggered based on the second trigger information.

If the first trigger information is non-eye information, then determine whether the non-eye information meets the preset requirements, and if so, determine which of the plurality of controlled devices is to be triggered based on the non-eye information the charged device;

judging whether the pupils of the person are in focus;

determining the controlled device to be triggered in response to the pupil focus of the person;

or,

judging whether the eye movement is a preset movement based on the eye movement information;

And determine the controlled device to be triggered based on the judgment result.

Optionally, judging whether to trigger the controlled device to be triggered based on the second trigger information includes:

judging whether the pupils of the person are in focus;

Responding to the pupil focus of the person, triggering the controlled device to be triggered;

or

If yes, trigger the controlled device to be triggered.

Optionally, after the controlled device is triggered, the controlled device prompts the user that the controlled device to be triggered has been activated or awakened in any one or more ways of voice, text, picture, sound and light change, and vibration .

In a second aspect, the present invention provides a target control device, the device comprising:

A first trigger information acquiring module, configured to acquire first trigger information in a preset area around the target, where the first trigger information includes eye information and/or non-eye information of people in the preset area;

A first trigger information judging module, configured to determine a controlled device to be triggered among the plurality of controlled devices based on the first trigger information;

A second trigger information acquisition module, configured to acquire second trigger information in a preset area around the target, where the second trigger information includes eye information and/or non-eye information of people in the preset area;

A second trigger information judging module, configured to judge whether to trigger the controlled device to be triggered based on the second trigger information;

A trigger information response module, configured to trigger the controlled device to be triggered based on the trigger information judgment result.

In a third aspect, the present invention provides a target control device, the device includes an information collection device, a memory, and a processor, the information collection device is used to obtain trigger information of the device, and machine-executable instructions are stored in the memory , when the machine-executable instructions are executed by the processor, the device is enabled to implement the target control method described in any one of the first aspects.

In a fourth aspect, the present invention provides a computer storage medium, which stores a computer program, and is characterized in that, after the computer program is executed, the target control method described in any one of the first aspects can be realized.

Beneficial technical effects:

In the case of adopting the above technical solution, the present invention can conveniently and accurately lock and trigger the controlled device to be triggered, thereby improving user experience.

Description of drawings

Describe preferred embodiment of the present invention below in conjunction with accompanying drawing, in the accompanying drawing:

Fig. 1 is the flow chart of the main steps of Embodiment 1 of the target control method of the present invention;

Fig. 2 is a flow chart of the main steps of Embodiment 2 of the target control method of the present invention;

Fig. 3 is a flow chart of the main steps of Embodiment 3 of the target control method of the present invention;

Fig. 4 is a flow chart of the main steps of Embodiment 4 of the target control method of the present invention;

Fig. 5 is a flow chart of the main steps of Embodiment 5 of the target control method of the present invention;

Fig. 6 is a schematic structural diagram of an embodiment of the target control device of the present invention.

Detailed ways

Some embodiments of the present invention are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present invention, and are not intended to limit the protection scope of the present invention.

In the description of the present invention, "device", "module" and "processor" may include hardware, software or a combination of both. A device or module may include hardware circuits, various appropriate sensors, communication ports, memory, and may also include software parts, such as program codes, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor or any other suitable processor. The processor has data and/or signal processing functions. The processor can be implemented in software, hardware or a combination of both. The non-transitory computer readable storage medium includes any suitable medium that can store program code, such as magnetic disks, hard disks, optical disks, flash memory, read only memory, random access memory, and the like.

In the first aspect, the present invention provides a target control method, the target includes multiple controlled devices, as shown in Figure 1, the present invention includes the following steps,

S1. Acquiring first trigger information in a preset area around the target, where the first trigger information includes eye information and/or non-eye information of people in the preset area;

S2. Determine a controlled device to be triggered among the plurality of controlled devices based on the first trigger information;

S3. Obtain second trigger information in a preset area around the target, where the second trigger information includes eye information and/or non-eye information of people in the preset area;

S4. Determine whether to trigger the controlled device to be triggered based on the second trigger information;

S5. Trigger the controlled device to be triggered based on the trigger information judgment result.

The content of the invention of the present invention will be described in detail below through specific examples.

In Embodiment 1 of the target control method of the present invention, as shown in FIG. 1 ,

The step S1 specifically includes acquiring first trigger information in a preset area around the target, where the first trigger information includes eye information and/or non-eye information of people in the preset area;

Specifically, the first trigger information in the preset area is acquired through an information collection device, and the first trigger information includes eye information and/or non-eye information of people in the preset area;

Wherein, the information collection device may be a camera, a video camera, or a camera with a pupil focusing function and/or a gesture information capture and/or body information capture function; it may also be a voice information collection device, such as a microphone or a pickup.

The preset area is an information collection area determined by debugging, testing and other means before use, and the area where user trigger information can be accurately, quickly and conveniently obtained by using the information collection device.

The eye information includes pupil information or eye movement information, wherein,

The pupil information may be pupil focus information, pupil size, pupil size change, pupil position, pupil position change.

The eye movement information may include blinking, the frequency of blinking, eye movement, eye movement direction, and eye movement trajectory.

The non-eye information includes any one or more of gesture information, body information, and voice information. For example, different gesture information can be represented by changes in hand movements, different body information can be represented by body changes, and different voice information can be represented by sounds or languages.

The step S2 specifically includes determining the controlled device to be triggered among the plurality of controlled devices based on the first trigger information; specifically, using the first trigger information obtained in the step S1 and the target internal preset The set corresponding relationship between the first trigger information and the controlled device determines the controlled device to be triggered among the plurality of controlled devices. That is, the acquired first trigger information can determine which controlled device the user wants to specifically trigger among the plurality of controlled devices.

Specifically, by judging whether the pupil of the person is in focus, and responding to the focus of the pupil of the person to determine the controlled device to be triggered; If the staring time is short, it means the first controlled device, and if the staring time is long, it means the second controlled device. It is also possible to further represent different devices according to different duration ranges. For example, gazing for 1-5 seconds indicates the first controlled device, gazing for 6-10 seconds indicates the second controlled device, and so on for more controlled devices. Or, judge whether the eye motion is a preset motion based on the eye motion information; and determine the controlled device to be triggered based on the judgment result. For example, the first controlled device is indicated by blinking fast and the second controlled device is indicated by slow blinking, or the first controlled device is indicated by blinking once and the second controlled device is indicated by blinking twice.

Specifically, different gesture information and body information of the user obtained through the camera may represent different controlled devices according to different gesture information and body information. It is also possible to obtain different voice information of the user through the voice collection device, and different voice information indicates different controlled devices.

For example, there are 3 controlled devices in total, which are controlled device A, controlled device B, and controlled device C. You can preset gesture 1 to indicate the operation of the controlled device A, and gesture 2 to indicate the operation of the controlled device B. , Gesture 3 means operating the controlled device C. Alternatively, by setting up in advance, raising the left hand means operating the controlled device A, raising the right hand means operating the controlled device B, and raising both hands means operating the controlled device C. Alternatively, by presetting the voice 1 means operating the controlled device A, the voice 2 means operating the controlled device B, and the voice 3 means operating the controlled device C.

It should be noted that, as a special case, if there is only one controlled device, there is no need to judge the number of controlled devices and the corresponding relationship, and the steps S1 and S2 can be omitted. The controlled device is directly triggered through steps S3, S4, and S5.

The step S3 is specifically, acquiring second trigger information in a preset area around the target, where the second trigger information includes eye information and/or non-eye information of people in the preset area;

Specifically, the content is the same as that of acquiring the first trigger information in the preset area around the target in the above-mentioned step S1, and the trigger information in the preset area around the target is also acquired through the information collection device. The manner of acquiring the second trigger information is the same as the manner of acquiring the first trigger information, which will not be repeated here.

The step S4 specifically includes, based on the second trigger information, judging whether to trigger the controlled device to be triggered; specifically, judging whether the second trigger information corresponds to the controlled device to be triggered, and if so, Then trigger the controlled device to be triggered based on the second trigger information.

That is, use the second trigger information obtained in step S3 and the corresponding trigger relationship between the second trigger information preset in the target and the controlled device to determine whether the controlled device to be triggered among the plurality of controlled devices is be triggered, and even a specific triggering manner may be further determined, and the triggering manner may be any one of operations such as being started, being woken up, being shut down, or being hibernated. That is, the acquired second trigger information can determine which type of triggering action is specifically performed by the controlled device that the user wants to specifically trigger among the plurality of controlled devices. For example, open the palm to indicate that the controlled device is triggered to turn on, and clenched a fist to indicate that the controlled device is triggered to be turned off. Or further, gesture 5 is preset to indicate that the controlled device is turned on, gesture 6 indicates that the controlled device is woken up, gesture 7 indicates that the controlled device is turned off, and gesture 8 indicates that the controlled device is sleep. Or, voice 1 means to turn on the controlled device, voice 2 means to wake up the controlled device, voice 3 means to turn off the controlled device, and voice 4 means to sleep the controlled device. By judging the second trigger information, the target can know whether the user wants to perform a trigger operation on the controlled device, and can further judge which type of trigger operation the specific user wants to perform.

The step S5 is specifically, based on the trigger information judgment result, triggering the controlled device to be triggered. Specifically, according to the judgment result of the above-mentioned step S4, the corresponding trigger operation is performed on the determined controlled device that needs to perform the trigger operation, so that the controlled device performs corresponding power-on, wake-up, or Actions such as shutting down or hibernating. After the controlled device is triggered, the controlled device executes the corresponding triggering action according to the received trigger command, and the controlled device can use any one or more methods of voice, text, picture, sound and light changes, and vibration Prompting the user that the controlled device to be triggered has been activated, or awakened, or shut down, or sleeps.

It should be noted that, in the present invention, if the trigger type is selected based on the judgment result of the first trigger information; the controlled device to be triggered among the multiple controlled devices is determined based on the second trigger information; such technology The scheme also belongs to the protection scope of the present invention.

In Embodiment 2 of the target control method of the present invention, as shown in FIG. 2 ,

For example, there are two controlled devices, controlled device 1 is an air conditioner, and controlled device 2 is a television.

Pre-set in the target to detect that the user looks up for more than 3 seconds to trigger the air conditioner, and the user looks forward for more than 3 seconds to trigger the TV; Gesture 1 means to turn on the device, and gesture 2 means to turn off the device .

First, the gaze direction of the user's eyes in the preset area around the target is obtained through the camera,

If it is detected that the user is looking up and lasts for more than 3 seconds, it is detected whether the user has made a preset gesture,

If user gesture 1 is detected, the operation of turning on the air conditioner is automatically triggered.

If it is detected that the user is looking forward and lasts for more than 3 seconds, it is detected whether the user has made a preset gesture,

If user gesture 2 is detected, the operation of turning off the TV is automatically triggered.

In the third embodiment of the target control method of the present invention, as shown in Figure 3,

It is pre-set in the target to detect user gesture 1 to trigger the air conditioner, gesture 2 to trigger the TV, the user to look up for more than 3 seconds to trigger the on operation, and the user to look forward for more than 3 seconds to trigger the shutdown operation.

Firstly, the change of the user's gesture in the preset area around the target is acquired through the camera,

If the user gesture 1 is detected, it is detected whether the user makes a preset eye movement,

If it is detected that the user looks up for more than 3 seconds, the operation of turning on the air conditioner is automatically triggered.

If the user gesture 2 is detected, it is detected whether the user makes a preset eye movement,

If it is detected that the user looks forward for more than 3 seconds, the operation of turning off the TV is automatically triggered.

In Embodiment 4 of the object control method of the present invention, as shown in FIG. 4 ,

It is pre-set in the target to detect that the user looks up and lasts for more than 3 seconds to trigger the air conditioner, the user looks forward for more than 3 seconds to trigger the TV, the user's eyeball rotates clockwise for one circle to turn on the operation, and the eyeball rotates counterclockwise for one circle to turn off operate.

First, obtain the gaze direction of the user's eyes in the preset area around the target through the camera, and if it is detected that the user is looking up and lasts for more than 3 seconds, then continue to detect whether the user makes a preset eye movement,

If it is detected that the user's eyeballs rotate clockwise for one circle, the operation of turning on the air conditioner is automatically triggered.

If it is detected that the user is looking forward and lasts for more than 3 seconds, continue to detect whether the user has made a preset eye movement,

If it is detected that the user's eyeballs rotate one circle counterclockwise, the operation of turning off the TV is automatically triggered.

In Embodiment 5 of the target control method of the present invention, as shown in FIG. 5 ,

In the target, it is pre-set that detecting user gesture 1 means triggering the air conditioner, user gesture 2 means triggering the TV, user voice 1 means turning on the operation, and voice 2 means turning off the operation.

If the user gesture 1 is detected, continue to detect whether the user makes a preset voice,

If the user's voice is detected to say 1, the operation of turning on the air conditioner is automatically triggered.

If user gesture 2 is detected, continue to detect whether the user makes a preset voice,

If it is detected that the user's voice says 2, the operation of turning off the TV is automatically triggered.

In a third aspect, the present invention provides a target control device 300. As shown in FIG. 6, the device 300 includes an information collection device 33, a memory 32, and a processor 31, and the information collection device 33 is used to obtain a trigger of the device information, the memory 32 stores machine-executable instructions, and when the machine-executable instructions are executed by the processor 31, the device 300 can realize the target control method described in any one of the first aspect .

So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings, but those skilled in the art will easily understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to relevant technical features, and the technical solutions after these changes or substitutions will all fall within the protection scope of the present invention.

Claims

A target control method, the target includes a plurality of controlled devices, characterized in that it includes the following steps,

Acquire first trigger information in a preset area around the target, where the first trigger information includes eye information and/or non-eye information of people in the preset area;

determining a controlled device to be triggered among the plurality of controlled devices based on the first trigger information;

Acquire second trigger information in a preset area around the target, where the second trigger information includes eye information and/or non-eye information of people in the preset area;

judging whether to trigger the controlled device to be triggered based on the second trigger information;

Based on the judgment result of the second trigger information, the controlled device to be triggered is triggered to start or wake up.
The target control method according to claim 1, wherein:

Acquiring trigger information in a preset area around the target includes acquiring eye information of the person, and/or acquiring non-eye information, wherein

The eye information includes pupil information or eye movement information,

The non-eye information includes any one or more of gesture information, body information, and voice information.
The target control method according to claim 1, wherein determining the controlled device to be triggered among the plurality of controlled devices based on the first trigger information comprises:

If the first trigger information is eye information, determine whether the eye information meets preset requirements, and if so, determine the controlled device to be triggered among the plurality of controlled devices based on the eye information ;

Judging whether to trigger the controlled device to be triggered based on the second trigger information includes:

Judging whether the second trigger information corresponds to the controlled device to be triggered, and if so, triggering the controlled device to be triggered based on the second trigger information.
The target control method according to claim 1, wherein determining the controlled device to be triggered among the plurality of controlled devices based on the first trigger information comprises:

If the first trigger information is non-eye information, then determine whether the non-eye information meets the preset requirements, and if so, determine which of the plurality of controlled devices is to be triggered based on the non-eye information the charged device;

Judging whether to trigger the controlled device to be triggered based on the second trigger information includes:

Judging whether the second trigger information corresponds to the controlled device to be triggered, and if so, triggering the controlled device to be triggered based on the second trigger information.
The target control method according to claim 1, wherein determining the controlled device to be triggered among the plurality of controlled devices based on the first trigger information comprises:

judging whether the pupils of the person are in focus;

determining the controlled device to be triggered in response to the pupil focus of the person;

or,

judging whether the eye movement is a preset movement based on the eye movement information;

And determine the controlled device to be triggered based on the judgment result.
The target control method according to claim 1, wherein judging whether to trigger the controlled device to be triggered based on the second trigger information comprises:

judging whether the pupils of the person are in focus;

Responding to the pupil focus of the person, triggering the controlled device to be triggered;

or

judging whether the eye movement is a preset movement based on the eye movement information;

If yes, trigger the controlled device to be triggered.
The target control method according to claim 1, wherein after the controlled device is triggered, the controlled device prompts the user that the The controlled device to be triggered has been activated or awakened.
A target control device, characterized in that the device comprises:

A first trigger information acquiring module, configured to acquire first trigger information in a preset area around the target, where the first trigger information includes eye information and/or non-eye information of people in the preset area;

A first trigger information judging module, configured to determine a controlled device to be triggered among the plurality of controlled devices based on the first trigger information;

A second trigger information acquisition module, configured to acquire second trigger information in a preset area around the target, where the second trigger information includes eye information and/or non-eye information of people in the preset area;

A second trigger information judging module, configured to judge whether to trigger the controlled device to be triggered based on the second trigger information;

A trigger information response module, configured to trigger the controlled device to be triggered based on the trigger information judgment result.
A target control device, characterized in that the device includes an information collection device, a memory, and a processor, the information collection device is used to obtain trigger information of the device, and machine-executable instructions are stored in the memory, when the When the machine-executable instructions are executed by the processor, the device can realize the target control method according to any one of claims 1-7.
A computer storage medium, the storage medium stores a computer program, characterized in that, after the computer program is executed, the target control method according to any one of claims 1 to 7 can be realized.