JP2021033978A - Virtual key board (vr-keyboard, virtual keyboard, air keyboard, ar-keyboard, air keyboard input method) for assisting realization of air instrument making sound and air controller as substitute for controller for operating game, radio controlled car, robot, construction machine, etc - Google Patents

Abstract

To prevent spoiling of diversity in input method by being incapable of adapting without increase in devices when responding to commercialization is not possible for a keyboard input method eliminating a keyboard (air-keyboard input method) in elimination of the keyboard requested from the points of marine pollution problem by plastic waste and waste export to a poorer country problem.SOLUTION: A virtual keyboard supports commercialization of a key board input method eliminating the keyboard (air keyboard input method) by reducing an occurrence of deviation between what a user intends (intention) and determination by an electronic equipment (recognition) by enabling correction in deviation between a home position in a brain and a home position outside the brain by having relative position relationship be capable of being confirmed through mapping capable of being semi-transparent and brought into a thumbnail realized on a monitor (synthesized on the monitor).SELECTED DRAWING: Figure 1

Description

The present invention relates to the input and modification of electronic devices such as electronic computers including tablet PCs, information terminals including smart phones, game machines with monitors, etc., and realizes air instruments that produce sound, games, radio controllers, etc. It is about an air controller that replaces a controller that operates robots, construction machines, and so on.

With the shift to mobile, keyboard-less has progressed, and at the same time, the usability of the voice input method has improved dramatically. However, some people still find the keyboard input method to be easier to use than the voice input method.
To be on the safe side, I haven't discussed which input method is better. As long as it is a problem that has no solution as long as people and situations are diverse, the question of which input method is superior cannot be argued in the first place. For example, "Voice input is sufficient. Recycling costs are high due to the low proportion of metal, there is a need to reduce regulations on plastics due to marine pollution problems, and it is rational to abolish keyboards." There is a claim. However, the reason why the claim is felt is that it is premised on the fact that there are no people who have difficulty speaking, or that it is true that no reasonable accommodation is given. Only if you have. Of course, it is also a reality that there are some people who assume that they are different from such a reality, and some who think that it is true reasonable accommodation not to give reasonable accommodation.

That doesn't mean that keyboardlessness will never end. This is because marine pollution from plastic waste and waste exports to poorer countries have to be a problem.

The problem to be solved is that the variety of input methods will be impaired as the keyboard becomes less.

Use it in front of the camera, envisioning a familiar keyboard until you can touch it blindly, like a guitarist playing an air guitar, a pianist playing an air piano, or a drummer playing an air drum. The point is that there is a discrepancy between the intention (will) of the user and the judgment (recognition) of the electronic device side just by performing the operation for inputting the keyboard that the person is accustomed to inputting.

In response to the problem of marine pollution caused by plastic waste and the problem of exporting waste to poorer countries, the reduction target of plastic is being set internationally like artificial carbon dioxide.

A keyboard input method that does not require a keyboard (air keyboard input method) that also supports the realization of air instruments that produce sound, and the realization of air controllers that can replace controllers that operate games, radio controllers, robots, construction machinery, etc. ).

Which key does the user intend to press?
About that, from the movement (change in position) of something that the camera captured (or as always) pressing (or pressing) the keyboard of the hand, foot, artificial hand, etc., the computer including the tablet PC , Information terminals including smart phones, game consoles with monitors, and other electronic devices should make the decision.

Correct the discrepancy between the intention (will) of the user and the judgment (recognition) of the electronic device through the translucent and thumbnail-ized mapping that appears on the monitor (combined on the monitor) (combined on the monitor). (Increase the synchronization rate).
The translucent and thumbnail images that appear on the monitor (combined on the monitor) are the hands, feet, and artificial hands that the electronic device that is compositely displayed on the monitor recognizes through the camera. The final communication tool for the user's electronic device, and the air keyboard selected by the user (= the keyboard that the user has called into the electronic device and intends to use = the completely transparent keyboard = It is the relative positional relationship with the keyboard (keyboard in the brain) that you are accustomed to until you can touch the blind you envision.

By hitting the fake keyboard printed on the paper, bag, cover, case, etc. captured by the camera, there is a gap between the intention (will) of the user and the judgment (recognition) of the electronic device. Try to reduce the rate. Increase the synchronization rate between the user and the electronic device.

By using the situation where the user is actually inputting using the real keyboard that he / she wants to use, and the result that AI has learned through the camera, the intention (will) of the user and the judgment (recognition) of the electronic device side. Try to correct the gap between. Increase the synchronization rate between the user and the electronic device.

As for the return key, the first return key confirms, and the second return key sets the decision as the initial setting, so that there is a discrepancy between the intention (will) of the user and the judgment (recognition) of the electronic device side. Try to reduce the incidence of. Increase the synchronization rate between the user and the electronic device.

The virtual keyboard of the present invention (VR keyboard, virtual keyboard, air keyboard, AR keyboard) enables input with the same intention and operation as when inputting with a keyboard, even without a keyboard. There are advantages.

Among them, the keyboard input method that does not require a keyboard using a camera may require fewer devices than those, so we used a method that applied termin, a method that applied laser communication, and a ring with a gravity sensor. There is an advantage that the hurdle for commercialization is lower than the method, the method of reading the surface myoelectric potential, and the method of reading the brain wave and the blood flow in the brain (brain machine interface).

Even if the real keyboard breaks, if there is no apparent problem, you have to go through the camera, but there is an advantage that you can continue keyboard input.

There is an advantage that it is possible to realize a keyboard-less society without impairing the variety of input methods.

There is an advantage that it is possible to contribute to the reduction of plastic waste without impairing the variety of input methods.

The voice input method, the conventional keyboard input method, and the air keyboard input method of the present invention have the advantage that the air keyboard input method of the present invention has the lowest risk of input information leakage.
The risk of input information leakage through the Internet is similar to that of the air keyboard input method of the present invention in both the voice input method and the conventional keyboard input. This is because it is determined by the hacking countermeasures of the OS, the security software used, and the Internet connection time.
However, when it comes to the risk of input information leakage by a classic wiretapping device, the voice input method is the highest.
The risk of input information leakage by the weak electromagnetic wave receiver that occurs when the keyboard is pressed is highest with the conventional keyboard input.
Therefore, since the input method does not work if the camera is physically disabled, the characteristics of the final transmission tool for the user (input person)'s electronic devices such as hands, feet, artificial hands, etc., and some of the desks, The air keyboard input method of the present invention has the highest risk of information leakage of clothes, rooms, etc., but the air keyboard input method of the present invention has the lowest input information leakage risk among these three methods. become.

There is an advantage that it may be useful to correct the deviation between the intention (will) of the user and the judgment (recognition) on the electronic device side in the input method using the eyeball and the eyelid captured by the camera.

There is an advantage that so-called healthy people may be able to understand the discomfort behind the joy of those who have experienced phantom limb pain and have recovered their functions with a type of artificial hand that reads the surface myoelectric potential. is there.

There is an advantage that so-called healthy people may be able to understand the discomfort behind the joy of those who have experienced phantom limb pain and have recovered their functions with a type of artificial hand that reads the surface myoelectric potential. is there.

It has the advantage that it may be useful for rehabilitation of cerebral infarction and learning how to play musical instruments.

It has the advantage of being able to cause philosophical controversy.
However, avatars are off the mark.

It also has the advantage of supporting the realization of air instruments that produce sound, and the realization of air controllers that can replace controllers that operate games, radio-controlled models, robots, construction machinery, and so on.

Figure 1 shows the type of keyboard that the user has called into the electronic device and intends to use (the keyboard in the brain, the keyboard that he or she is accustomed to until the blind touch that he or she envisions), and the user that the electronic device recognizes through the camera. The final communication tool for electronic devices (the type of hand, foot, prosthesis, etc. that was pressing the keyboard until now (or as always) that the model intends to use, the model is the right hand prosthesis with the mouse operated with the left hand) , Is an explanatory diagram showing their relative positional relationship synthesized on the monitor. (Example 1) Figure 2 shows the type that the user calls into an electronic device for the type that makes the virtual keyboard (VR keyboard, virtual keyboard, air keyboard, AR keyboard, air keyboard input method) feel uncomfortable. The type of keyboard you intend to use (the keyboard in your brain, the keyboard you are accustomed to until you can touch the blinds you envision), the intention of the user recognized by the electronic device through the camera (will or deviation may have occurred) Something entered that may not have occurred), is an explanatory diagram of the specification that only shows their relationship synthesized on the monitor. (Example 2) FIG. 3 is an explanatory diagram of a state in which AI is trained, but is an explanatory diagram of a method of inputting a keyboard through a camera while recognizing a broken real keyboard that has no apparent problem. (Example 3) (Example 6) FIG. 4 is an explanatory diagram of a method of inputting a keyboard through a camera while recognizing a fake keyboard printed on paper, a bag, a cover, a case, etc., but an explanation of a state in which AI is learning. It is also a figure. (Example 5) (Example 7) FIG. 5 is an explanatory diagram of an embodiment (piano, organ) of an air musical instrument that produces sound. (Example 8) FIG. 6 is an explanatory diagram of an embodiment (piano, organ) of an air musical instrument that produces sound so that a beginner can practice by using a fake musical instrument printed on paper, a bag, a cover, a case, or the like. (Example 9) FIG. 7 is an explanatory diagram of one embodiment (piano, organ) of a self-study boosting method using a second mapping that allows the role of a trainer or a coach. (Example 10) FIG. 8 is an explanatory diagram of one embodiment (drone) of the simple simulator to which the first embodiment is applied. (Example 11)

A method that applies thermin, a method that applies laser communication, a method that uses a ring with a gravity sensor, a method that reads surface myoelectric potential, a method that reads brain waves and cerebral blood flow (brain machine interface), etc. Realize a keyboard input method that does not require a keyboard without using an unfinished dedicated device.

Blind touch is possible.
The meaning is diverse, but it has the following meaning in relation to the visual recognition of the user (practitioner of blind touch). A person who is touch-typing is a person who is typing using a fully transparent keyboard.
So what does it mean in relation to face recognition AI and its researchers? It's like this. A blind touch person is someone who provides the indirect evidence needed to visualize a fully transparent keyboard from test input and finger movements.
Therefore, when touch-typing is done, a keyboard input method that does not require a keyboard is possible through the camera.
By the way, this is also the reason why it is more accurate to say "air keyboard input method" than to say "keyboard input method like avatar". When you look at the explanatory diagram in Fig. 1 or commercialize it, some people may want to say, "It looks like an avatar."

That said, blind touch accuracy can be reduced.
Of course, the possible causes are also diverse. Attacked by illness such as cerebral infarction, attacked by sleep, attacked by hackers and viruses, completely transparent keyboard changed model, camera failed, (external blind touch accuracy) This is one item that the authentication system AI and its researchers must check because there are times when you do not know everything you want to enter before it is entered. The content of what you want to enter has changed (although it is not actual), and so on.
However, when the deadline is approaching, there is an urgent need, the schedule has been changed, the desk or chair has changed, the shoulders have become stiff and the posture has to be changed, etc. If you do not take measures to improve the problem of poor blind touch accuracy during air keyboard input, you will not be able to withstand commercialization.
Therefore, it is essential for the air keyboard input method to incorporate a method for correcting the discrepancy between the intention (will) of the user and the judgment (recognition) of the electronic device side.

Therefore, it is FIG.
FIG. 1 is an explanatory diagram of an embodiment relating to a method of correcting a deviation between a user's intention (will) and a judgment (recognition) on the electronic device side in the air keyboard input method of the present invention.
The type of keyboard that the user has called into the electronic device and intends to use (= fully transparent keyboard = the keyboard that he or she is accustomed to until the blind touch that he or she envisions = the keyboard in the brain = the air keyboard). The model with the hand, foot, prosthesis, etc. that the electronic device recognized through the camera as the final communication tool to the user's electronic device (or the hand, foot, prosthesis, etc. that was (or always) the keyboard was pressed until now). Is a type of right-handed prosthesis that operates the mouse with the left hand). By compositing and displaying their relative positional relationship on the monitor, it is possible to confirm the position and correct the deviation between the in-brain home position and the extra-brain home position. For example, the return key is the first time. The return key of is confirmed, the second return key is decided, and this is the initial setting.
This makes it possible to reduce the incidence of discrepancies between the user's intention (will) and the judgment (recognition) of the electronic device while using the air keyboard input method. It is possible to increase the synchronization rate between the user and the electronic device.

The embodiment shown in FIG. 2 is a final transmission tool for the user's electronic device recognized through the camera by the electronic device synthesized and displayed on the monitor of the first embodiment (FIG. 1) (intention to be used from now on). This model is for the type of hand, foot, prosthesis, etc. that has been pressing the keyboard until now (or as always), and the type in which the mouse is operated with the left hand and the right hand prosthesis is felt unpleasant.
In short, the type of keyboard that the user called into the electronic device and intends to use (= fully transparent keyboard = the keyboard that he or she is accustomed to until the blind touch that he or she envisions = the keyboard in the brain = the air keyboard) , It is a specification that only the intention (will) of the user while using the air keyboard input method judged (recognized) by the electronic device is compositely displayed on the monitor.

By the way, what if you want to use a model that could not be incorporated in advance because secondary use was not permitted? In such a case, as suggested by the explanatory diagram of the first embodiment of FIG. 3, if the electronic device is made to learn through the test input and the confirmation / deletion via the camera, the air keyboard input can be obtained. It becomes possible to realize the method.

Of course, through test input and confirmation / deletion via the camera, the type of keyboard that the user intends to use (= fully transparent keyboard = the keyboard that he is familiar with until he can touch the blinds he envisions = the keyboard in his brain = There is also a method of incorporating an air keyboard).
However, if the intracerebral keyboard in the test input stage and its output operation (= air keyboard input work in the test input stage) are not perfect, at least if it is not frequent, reset and go through the camera. You will have to redo the test entry and confirmation / deletion. If you don't start over, you'll end up with more stressful input work.

If the user calls the keyboard in the electronic device and intends to use it (= fully transparent keyboard = envisioned blind touch, familiar keyboard = intracerebral keyboard = air keyboard) is 1 A fake keyboard drawn on paper, bags, covers, cases, etc. on a 1 / scale scale, and a fake mouse (or mascot) drawn on paper, bags, covers, cases, etc. of the model you intend to use. What if there was a fake mouse, or a fake mouse that was stuffed, or a fake mouse that was mascotized, or a fake mouse that was keyheld?
It is possible to reduce misjudgment (misrecognition) on the electronic device side.
Therefore, the first embodiment shown in FIG. 4 is also one of the methods for correcting the discrepancy between the intention (will) of the user and the judgment (recognition) on the electronic device side while using the air keyboard input method. It means that.

As a matter of course, if there is a broken real keyboard that has no apparent problem, the intention (will) of the user and the judgment (recognition) of the electronic device side while using the air keyboard input method are as in the fifth embodiment. It will be one of the ways to correct the gap between them.

The keyboard that I was accustomed to until I could touch it blindly was one of the models that could not be incorporated in advance because secondary use was not permitted, and it has been a long time since the production was discontinued, and from the second-hand market. Even if it disappears and is lost, if there is a fake keyboard drawn on paper, a bag, a cover, a case, etc. on a scale that is one-third that of Example 5, the same as in Example 3. , It is possible to realize the air keyboard input method by having an electronic device learn through a camera.

By the way, what if the tone of the instrument is output to the air input? Furthermore, what happens if radio waves are transmitted like a drone controller?
It will be possible to realize an air instrument that produces sound, and an air controller that replaces the controller that operates games, radio-controlled models, robots, construction machinery, and so on.
One embodiment shown in FIG. 5 is one embodiment (piano, organ) of an air musical instrument that produces sound.

And what if there were fake instruments printed on paper, bags, covers, cases, etc.?
Beginners will be able to practice.
Of course, the slight time lag that occurs and the misery that will occur more or less are undeniably negative aspects. In addition, it is undeniably a downside that parents and children who are in a situation where they cannot even practice with fake instruments will suffer further.
However, if you have a computer with a camera and a fake keyboard printed on, for example, one-scale paper, you can practice the piano, which is a plus, at least in equal opportunity.
FIG. 6 is an explanatory diagram of an embodiment (piano, organ) of an air musical instrument that produces sound so that a beginner can practice by using a fake musical instrument printed on paper, a bag, a cover, a case, or the like.

A map that appears on the monitor (combined on the monitor) that can be made translucent and thumbnails, or a partial reorganization of the map (= makes it easier to recognize that you are not yourself in terms of visual information). The second map, which allows you to play the role of a trainer or coach who sets an example, shows an example that suits each situation, and the user follows it, so that rehabilitation through typing can be performed. , It will be possible to support the learning of how to play real musical instruments through fake musical instruments without a trainer or coach.
As a precaution, if the monitor can be divided into two so that the first map and the second map can be projected at the same time, the options can be expanded. Another reason is that the present invention can be translucent and thumbnailed, so even if you are playing an air instrument that produces sound with a fake instrument, you can use the existing piano self-study support software in principle. It is possible.
FIG. 7 is an explanatory diagram of one embodiment (piano, organ) of a self-study boosting method using a second mapping that allows the role of a trainer or a coach.

By the way, if the air input is changed, the virtual image captured by the drone-equipped camera and the virtual image of the arm such as the hydraulic excavator seen from the cockpit and the bucket attached to the tip of the arm appear on the monitor. What happens if it is made to be (combined on the monitor)?
In short, the drone-equipped camera captures the "type of keyboard that the user has called into the electronic device and intends to use (the keyboard in the brain, the keyboard that he is accustomed to until he can touch the blinds he envisions)" in Example 1. Replace with "video, etc." Then, "For example, with the hand, foot, artificial hand, etc. that was pressing the keyboard until now (or as always) that the electronic device recognized through the camera as the final communication tool for the user's electronic device (or as always). , The model is a type that operates the mouse with the left hand and the right hand prosthesis) "is replaced with" the degree and time of the control stick of the controller in the power off state (= air input for the virtual drone etc. in the simple simulator) " .. Then, "their relative positional relationship" is replaced with "their response relationship".
Become a simple simulator.
Of course, there is a simpler method (at least, since it is a utility model, its specific description is avoided), but the input method via a camera has the advantage of lowering the risk of malfunction. This is because if the input method is via a camera, it will be a simple simulator even when the power supply on the controller side is turned off.
FIG. 8 is an explanatory diagram of one embodiment (drone) of the simple simulator to which the first embodiment is applied.

Since it is also a method to realize a keyboard input method (air keyboard input method) that does not require a keyboard without using an unfinished dedicated device, a method that applies termin, a method that applies laser communication, and a gravity sensor are attached. Compared with the method using a ring, the method of reading the surface myoelectric potential, the method of reading the brain wave and the blood flow in the brain (brain machine interface), etc., the hurdle for commercialization of the present invention is low.

By making it possible to confirm the relative positional relationship through a map that can be made translucent and thumbnailed (combined on the monitor) that appears on the monitor, the deviation between the intracerebral home position and the extracerebral home position can be confirmed. Make corrections possible and reduce the rate of discrepancies between the user's intention (will) and the judgment (recognition) of the electronic device.
One element constituting the present invention is a method applying a theremin, a method applying laser communication, a method using a ring with a gravity sensor, a method of reading surface myoelectric potential, a method of reading brain waves and cerebral blood flow. (Brain machine interface), etc. may also contribute to the reduction of the deviation occurrence rate.

When the present invention is accepted and commercialized, the right of claim 3 of the present invention is waived to the keyboard manufacturing company which was established before acceptance or publication and actually put the product on the market. .. Therefore, even if the present invention further promotes keyboard-less operation, the keyboard production company can secure a preparation period for changing the direction.

Even if a reduction target for plastics is set internationally, such as artificial carbon dioxide, if the present invention is commercialized in advance, it will not be eliminated.

Replace the real keyboard and mouse in this case with a real musical instrument and a real controller. Replace a broken real keyboard with no apparent problems, a broken real mouse with no apparent problems with a broken real instrument with no apparent problems, or a broken real controller with no apparent problems. Replace fake keyboard and fake mouse with fake instruments and fake controllers. By doing so, it becomes possible to realize an air controller that replaces a controller that operates a sound-producing air musical instrument, a game, a radio control, a robot, a construction machine, and the like.

1 Monitor 2 Camera 3 The intention (will) of the user judged (recognized) by the electronic device (or something entered that may or may not be misaligned)
4 Something that has been (or always) been pressing (or is pressing) the keyboard of the hand, foot, artificial limb, etc., or something that was playing (or playing) an instrument 5 Real keyboard (or) Broken real keyboard with no apparent problems)
6 Real mouse (or broken real mouse that looks fine)
7 A map of something that the camera has ever (or always) pressed (or is pressing) on the keyboard of a hand, foot, artificial limb, etc., or something that was playing (or playing) an instrument. (Map of reference numeral 4)
8 The envisioned blind touch-familiar keyboard (or the selected keyboard model, or the real keyboard recognized by the AI you are learning, or the fake keyboard recognized by the AI you are learning, or A fake keyboard to assist in correcting the discrepancy between the user's intention (will) captured by the camera and the judgment (recognition) on the electronic device side, or an electrically broken, unconnected real captured by the camera. A mapping of the keyboard or unconnected real keyboard captured by the camera 9 The mouse envisioned (or the mode of the selected mouse, or the real mouse recognized by the AI being trained, or captured by the camera A fake mouse to assist in correcting the discrepancy between the user's intention (will) and the electronic device's judgment (recognition), or an electrically broken, unconnected real mouse captured by the camera, or the camera. A mapping of an unconnected real mouse captured by 10 10 The intention (will) (or deviation) of the user judged (recognized) by an undetermined electronic device may or may not have been entered. Blinking (or negative / positive reversal, discoloration, whiteout, or enormous) to inform the user of something)
11 Fake keyboard drawn on paper, bags, covers, cases, etc. 12 Fake mouse (or mascotized fake mouse, or stuffed fake) drawn on paper, bags, covers, cases, etc. -Mouse, mascotized fake mouse, or keychain fake mouse)
13 The type of the selected musical instrument (or the real musical instrument recognized by the AI being learned, the fake musical instrument recognized by the AI being learned, or the intention (will) of the user captured by the camera and the judgment of the electronic device side. Mapping of a fake instrument to assist in correcting the deviation between (recognition), or an electrically broken unconnected real instrument captured by the camera, or an unconnected real instrument captured by the camera 14 Real instrument ( Or a broken real instrument that has no apparent problem)
15 Fake musical instrument (or real musical instrument recognized by AI during learning, or fake musical instrument to assist correction of deviation between the intention (will) of the user captured by the camera and the judgment (recognition) of the electronic device side. )
16 A second map that can take on the role of a trainer or coach 17 Controller 18 Virtual image captured by a drone-equipped camera that appears on the monitor and changes according to the air input (synthesized on the monitor)

The present invention relates to an electronic computer also including tablet PC, information terminals that also includes smart phones, monitor with a game machine, is related to the input and modification of the like of electronic equipment, the realization of air instruments sound comes out, game, radio - those controls cars, robots, construction equipment, relates to an air controller that takes the place of the controller to operate the like.
(1) A method that allows keyboard input even with a fake keyboard printed on paper.
(2) A method that allows keyboard input without even a fake keyboard printed on paper.
(3) A method of applying the method to make a sound from a fake / musical instrument printed on paper.
(4) And further, how to realize a simple simulator.

With the shift to mobile, keyboard-less has progressed, and at the same time, the usability of the voice input method has improved dramatically. However, some people still find the keyboard input method to be easier to use than the voice input method.
To be on the safe side, I haven't discussed which input method is better. As long as people and situations are diverse, it becomes an unsolvable problem . As long as it is , the question of which input method is superior cannot be argued in the first place. For example, "Voice input is sufficient. Recycling costs are high due to the low proportion of metal, there is a need to reduce regulations on plastics due to marine pollution problems, and it is rational to abolish keyboards." There is a claim. However , the claim seems rational if (1) it is premised on the fact that there are no people who have difficulty speaking, or (2) "it is true that no reasonable accommodation is given." It is limited to those who think that it is reasonable , or one of them, or both. Of course, there are people who assume that they are different from such reality, people who think that "not reasonable accommodation is true reason", people who do not, and so on. It is also a reality.

That doesn't mean that keyboardlessness will never end. This is because (1) marine pollution caused by plastic waste and (2) export of waste to poorer countries are the two issues that have to be addressed.

A problem to be solved is that the keyboard-less is Ru diversity of input method is impaired if proceed.

Like a guitarist playing an air guitar, a pianist playing an air piano, a drummer playing an air drum, (1) envisioning a familiar keyboard until you can touch it blindly (2) ) What the user wants to input in front of the camera , (3) Perform the operation to input using the familiar keyboard , that alone is the intention (will) of the user and the judgment (recognition) of the electronic device side. it is that the deviation arising between.

(1) marine pollution problems caused by plastic waste and, receiving the waste export problem to the poor than (2) countries, such as artificial carbon dioxide, is at some point while internationally erected reduction targets of plastic.

Realization of air instruments sound comes out, game, radio - Control car, robot, construction machinery, also to boost the implementation of the air controller to become a place of the controller to operate and so on, keyboard input method that does not require a keyboard ( Air keyboard input method).

Unfinished devices such as rings with gravity sensors, keyboard input method that does not use them and does not require a keyboard (air keyboard input method).

Which key does the user intend to press?
About that, from the movement (change in position) of something that the camera captured ( or as always) pressing (or pressing) the keyboard of the hand, foot, artificial hand, etc., the computer including the tablet PC , information terminals that also includes smart phones, monitor with a game machine, an electronic device side, etc., to determine.

Correct the discrepancy between the user's intention (will) and the judgment (recognition) of the electronic device through the semi-transparent and thumbnail-ized mapping that appears on the monitor (combined on the monitor). (Increase the synchronization rate) .
The semi-transparent and thumbnailable mapping that appears on the monitor (combined on the monitor) is, that is, (1) the hand recognized by the electronic device compositely displayed on the monitor through the camera. The final communication tool for the user's electronic devices such as feet, artificial hands, etc., and (2) the air keyboard selected by the user (= the keyboard that the user called into the electronic device and intends to use = completely the clarified keyboard = envisioned by which keyboard = keyboard brain familiar until blind touch), the relative positional relationship between them.

By hitting the fake keyboard printed on the paper, bag, cover, case, etc. captured by the camera, the incidence of deviation between the user's intention (will) and the judgment (recognition) of the electronic device side can be determined. reduction Ru. Ru increase the synchro rate between the user and the electronic device.

Situations in which the user is entering the real keyboard actually using you want to use, by utilizing the result of it the AI has learned through the camera, the intention of the user (will) and of electronic equipment side judgment (recognition) that to correct the deviation between. Ru increase the synchro rate between the user and the electronic device.

The return key, (1) confirmed first time the return key, (2) the second time of the return key is determined by the initial set, the intention of the user (will) and the electronic equipment side Pulling the incidence of deviation between the judgment (recognition) of. Ru increase the synchro rate between the user and the electronic device.

The virtual keyboard (VR keyboard, virtual keyboard, air keyboard, AR keyboard) of the present invention has an advantage that it can be input with the same intention and operation as when inputting with a keyboard without a keyboard. is there.

Among them, the keyboard input method that does not require a keyboard using a camera may require fewer devices than those, so (1) a method that applies termin, (2) a method that applies laser communication, and (3). A method using a ring with a gravity sensor, (4) a method of reading surface myoelectric potential, (5) a method of reading brain waves and cerebral blood flow (brain machine interface), the hurdles for commercialization are lower than those. There is an advantage of.

Even if broken real keyboard, if there is no appearance problem, but must be going through a camera, can continue to keyboard input, there is an advantage in that.

Without compromising the diversity of input method, keyboard-less society is realized, there is an advantage that.

Without compromising the diversity of input method, it reduces the plastic waste, there is an advantage that.

(1) Voice input method, (2) Conventional keyboard input method, (3) Air keyboard input method of the present invention, among which the air keyboard input method of the present invention has the lowest risk of input information leakage. there, there is an advantage in that.
Regarding the risk of input information leakage through the Internet, both the voice input method and the conventional keyboard input are similar to the air keyboard input method of the present invention. This is because it is determined by (1) how to prevent hacking of the OS, (2) what kind of security software is used, and (3) how long the Internet is connected.
However, if the input information leakage risk by classical wiretap, speech input method most have high is.
Then, for the input of information leakage risk by weak electromagnetic wave receiver that occurs when pressing the keyboard, a conventional keyboard input is not the highest.
Therefore, since the input method does not work if the camera is physically disabled, (1) the characteristics of the final transmission tool for the user (input person)'s electronic device such as hands, feet, artificial hands, etc., and (2) desk or in part, clothing, etc. room but the highest their personal information leakage risk is air keyboard input method of the present invention, the input information leakage risk, air keyboard input method of the present invention most have low in the three methods.

In the input method by eye and eyelid the camera is captured, which may serve to modify the deviation between the intention of the user (will) and the electronic device side determines (recognizes), there is an advantage that.

And have the experience of phantom limb pain, a sense of discomfort in the back of the joy of the people who played a functional recovery in the type of artificial arm to read the surface muscle potential, there is a possibility that can be understood by so-called healthy person, there is an advantage in that.

And rehabilitation of cerebral infarction, Ru possibility there to help you learn how to play musical instruments, there is an advantage.

There is a possibility that stir up philosophical debate, there is an advantage in that.
However, avatars are off the mark.

Realization of air instruments sound comes out, game, radio - Control car, robot, construction machinery, also to boost the implementation of the air controller to become a place of the controller to operate and so on, there is an advantage in that.

Figure 1 shows (1) the type of keyboard that the user has called into the electronic device and intends to use (the keyboard in the brain, the keyboard that he is accustomed to until he can touch the blinds he envisions), and (2) the electronic device uses the camera. The model operates the mouse with the left hand, with the hand, foot, prosthesis, etc. that was pressing the keyboard until now (or as always) the final communication tool to the user's electronic device recognized through. It is a type of right-handed prosthesis), (3) their relative positional relationship synthesized on the monitor, and an explanatory diagram about them. (Example 1) FIG. 2 is a virtual keyboard (VR · keyboard, virtual keyboard, air keyboard, AR · keyboard, air-keyboard input method) for the type of Ru uncomfortable to, (1) call the user is in an electronic equipment The type of keyboard that you intend to use (keyboard in your brain, keyboard that you are familiar with until you can touch the blinds you envision), (2) The intention of the user that the electronic device recognized through the camera (will or deviation occurs) something which might not to have been entered may not have occurred may have), their relationship is an explanatory diagram of distillate circle specification Ru synthesized on the monitor. (Example 2) FIG. 3 is an explanatory diagram of a state in which AI is trained, but is an explanatory diagram of a method of inputting a keyboard through a camera while recognizing a broken real keyboard that has no apparent problem. (Example 3) (Example 6) 4, paper or bags, covers, cases, while also recognize the printed fake keyboard etc., but an illustration of the method of keyboard input via the camera, the information about how you are programming the on AI It is also an explanatory diagram. (Example 5) (Example 7) FIG. 5 is an explanatory diagram of an embodiment (piano, organ) of an air musical instrument that produces sound. (Example 8) 6, paper and bags, covers, cases, the fake instrument printed on etc., the information about how to air instruments sound comes out is an explanatory view of one embodiment (piano, organ). (Example 9) 7, for self-taught boost method using a second mapping that Ninae roles trainers and coaches, an illustration of one embodiment (piano, organ). (Example 10) FIG. 8 is an explanatory diagram of one embodiment (drone) of the simple simulator to which the first embodiment is applied. (Example 11)

(1) Method applying termin, (2) Method applying laser communication, (3) Method using ring with gravity sensor, (4) Method of reading surface myoelectric potential, (5) Brain wave and brain A keyboard input method that does not require a keyboard is realized without using an unfinished dedicated device such as a method for reading blood flow (brain machine interface).

Blind touch is possible.
The meaning is diverse, but it has the following meaning in relation to the visual recognition of the user (practitioner of blind touch). A person who is touch-typing is a person who is typing using a fully transparent keyboard.
So what does it mean in relation to face recognition AI and its researchers? It's like this. A blind touch person is someone who provides the indirect evidence needed to visualize a fully transparent keyboard from test input and finger movements.
Therefore, in the case of blind touch, a keyboard input method that does not require a keyboard can be realized by using a camera.
By the way, this is, rather than "Avatar like a keyboard input method" also, there is also the reason why those who say "air-keyboard input method" is more accurate. When you look at the explanatory diagram in Fig. 1 or commercialize it, some people may want to say, "It looks like an avatar."

That said, blind touch accuracy can be reduced.
Of course, the possible causes are also diverse. (1) Attacked by illness such as cerebral infarction, (2) Attacked by sleep, (4) Attacked by hackers and viruses, (5) The model of the completely transparent keyboard changed, (6) The camera broke down, (7) (The accuracy of the external blind touch had to be an issue, and sometimes I didn't know everything I wanted to enter before it was entered, so the authentication system. even if there is one item that is AI and the side of the researchers must check, but is not a substantive) has changed the contents of the things you want to enter, and so on.
However, (1) the deadline was approaching, (2) urgent need was made, (3) schedule was changed, (4) desks and chairs were changed, (5) shoulder stiffness became severe and the posture had to be changed. Banara was not, in the case caused by the pressure and fatigue, etc., if not taken measures to improve the problem of blind touch of accuracy decreases in air-keyboard input, it will not withstand the commercialization ..
Therefore, air keyboard input method, virtually built way to correct the deviation between the intention of the user (will) and the electronic device side determines (recognizes) is essential.

Therefore, it is FIG.
1, the air keyboard input method of the present invention, the information on how to correct the deviation between the intention of the user (will) and the electronic device side determines (recognizes) an explanatory view of one embodiment is there.
(1) The model of the keyboard that the user has called into the electronic device and intends to use it (= completely transparent keyboard = envisioned blind touch-familiar keyboard = intracerebral keyboard = air keyboard) .. (2) The final communication tool for the user's electronic device recognized by the electronic device through the camera (with the hand, foot, artificial hand, etc. that was pressing the keyboard until now (or as always) that the electronic device intends to use). , The model is a type in which the mouse is operated with the left hand and the right hand is prosthesis). (3) The relative positional relationship between them is displayed compositely on the monitor . Thus, to check the position, to modify the deviation or the like between the brain home position and extracerebral home position. Then , for example, for the return key , the first return key is confirmed, the second return key is determined, and this is the initial setting.
It's in this, and how to reduce the incidence of the deviation between the intention of the user during use of the air-keyboard input method (will) and of electronic equipment side judgment (recognition). A method of increasing the synchro rate between the user and the electronic device.

The embodiment shown in FIG. 2 is a final transmission tool for the user's electronic device recognized through the camera by the electronic device synthesized and displayed on the monitor of the first embodiment (FIG. 1) (intention to be used from now on). For the type of hand, foot, prosthesis, etc. that has been pressing the keyboard until now ( or as always), the model is a type of right hand prosthesis with the mouse operated with the left hand) , which feels unpleasant. is there.
In short, (1) The model of the keyboard that the user has called into the electronic device and intends to use it (= fully transparent keyboard = the blind touch that he envisions, the keyboard that he is familiar with until he can touch it = the keyboard in his brain = air. and keyboard), (2) electronic devices is determined (recognition) and the intention of the user during use of the air-keyboard input scheme (will), only the relationship is a specification that will be synthesized and displayed on a monitor.

By the way, what if you want to use a model that could not be incorporated in advance because secondary use was not permitted? In such a case, as suggested from the illustration of an embodiment of FIG. 3, the electronic apparatus, through the established / deleted and the test input via the camera, cause learning. That way, the air keyboard input method can be realized.

Of course, only by test input and confirmation / deletion through the camera, the model of the keyboard that the user intends to use from now on (= fully transparent keyboard = the keyboard that he is accustomed to until he can touch the blind that he envisions = in his brain There is also a method of incorporating a keyboard (air keyboard).
However, in that case , if the intracerebral keyboard at the test input stage and its output operation (= air keyboard input work at the test input stage) are not perfect, at least if it is not frequent, reset the camera. You will have to redo the test input and confirmation / deletion via. If not start over if, ing to the input work while feeling more stress until now. Because the incidence of deviation increases. Because the sync rate goes down.

If the user called it into an electronic device , (1) the type of keyboard that he intends to use (= fully transparent keyboard = the blind touch that he envisions, the keyboard that he is familiar with until he can touch it = the keyboard in his brain = air of the keyboard), 1 minute 1 scale of paper and bags, cover, case, and fake keyboard drawn on and so on, (2) of the type you are in going to use from now on, 1 minute 1 scale of paper and bags, Fake mice drawn on covers, cases, etc. (or mascotized fake mice, or stuffed fake mice, or mascotized fake mice, or keyheld fake mice) , What if they were there?
To reduce the electronic device side of misjudgment (false recognition).
Therefore, one embodiment shown in FIG. 4 also of a method for modifying the deviation between during use of the air-keyboard input method, the intention of the user (will) and the electronic device side determines (recognizes) It is one.

Naturally, the real keyboard was broken there is no appearance problem also, Example 5 Similarly, in the use of air-keyboard input method, the intention of the user and (will) the electronics side of the determined that (recognition) It is one of the methods to correct the gap between.

The keyboard that I was accustomed to until I could touch it blindly was one of the models that could not be incorporated in advance due to reasons such as (1) secondary use was not permitted, and (2) a long time after production was discontinued. is passed, it was erased the figure from (3) used market. Even if you lose the keyboard you are accustomed to until you can touch it blindly in that state, if you have a fake keyboard drawn on paper, a bag, a cover, a case, etc. on a scale of 1/1 of Example 5, Example 3 Similarly, by learning through the camera to the electronic device, an air-keyboard input method can be implemented.

By the way, If you like the sound of musical instruments to the air input is output, what happens. In addition, if you like radio waves are transmitted to the drone of the controller, what happens.
(1) sound comes out air musical instrument, and (2), games, radio - Control car, robot, construction machinery, air-controller which is in place of the controller to operate and so on, can realize it, et al.
One embodiment shown in FIG. 5 is one embodiment (piano, organ) for an air musical instrument that produces sound.

In addition, if the paper and bags, there was cover, case, the fake musical instrument that has been printed on and so on, what happens.
Beginners can practice the piano without the piano .
Of course, the slight time lag that occurs and the misery that will occur more or less are undeniably negative aspects. In addition, the parents in the shuffle circumstances even to practice with faux instrument, also would become further plagued a downside undeniably.
However, (1) camera with a personal computer or the like, (2) for example, one minute of 1 scale of the printed fake keyboard on paper, if there is them, can practice the piano, this is if at least from the point of view of equal opportunities, plus It is a face.
FIG. 6 is an explanatory diagram of an embodiment (piano, organ) of an air musical instrument (piano, organ) that produces a sound that can be practiced by beginners by using a fake musical instrument printed on paper, a bag, a cover, a case, etc. is there.

(1) was revealing on a monitor (synthesized on the monitor) translucent be thumbnailed also possible or mapping, (2) that (= not visual information to their by reform partially the mapping second mapping which is adapted Ninaeru the role of trainers and coaches showing a) model by easily to have a recognition, to that someone, Ru was shown a model that matches the respective circumstances. The user follows it . As a result, (1) rehabilitation and through the typing, (2) the acquisition of real musical instrument method of through the fake musical instruments, trainers and coaches of Shino boost to achieve.
As a precaution, according to taste, the first mapping and the second mapping is out reflects at the same time is divided into two monitors, In this way, the choices want wide. Another, therefore the present invention can also be a thumbnail of semi-transparent, even if you're playing the air instruments sound fake musical instrument comes out, the existing piano self-taught support software is available.
Figure 7 is a second mapping which is adapted Ninaeru the role of trainers and coaches, for self-taught boost method using the same, is an explanatory view of one embodiment (piano, organ).

By the way, if the air input is (1) a virtual image captured by a drone-equipped camera that changes accordingly, or (2) an arm such as a hydraulic excavator that changes accordingly and is attached to the tip of an arm such as a hydraulic excavator. Virtual image of the bucket, is to appear on the monitor (which is synthesized on a monitor), what happens.
In short, (1) the "type of keyboard that the user is present with the intention used from which that called in the electronic device (intracerebral keyboard, keyboard familiar until blind touch envision)" in Example 1, "drone mounted Replace with " image captured by the camera, etc." (2) Then, " For example, the hand, foot, which the electronic device recognized through the camera, was pressing the keyboard as the final communication tool (as ever (or anytime) that the electronic device intends to use) to the user's electronic device. With a prosthetic hand, etc., the model is a type in which the mouse is operated with the left hand and the right hand is prosthetic hand) " , " The degree and time of the control stick of the controller in the power off state (= air input to the virtual drone etc. in the simple simulator ) ” . (3) and further to "their relative positional relationship" is replaced with "their response relationships."
A simple simulator is realized .
Of course, there are more simple method (the specific description because it also becomes Utility Model Low avoid) also, the input method via the camera, there is an advantage to be lowered malfunction risk than that. If the input method via the camera, even when you turn off the controller side is because is simplified simulator.
Figure 8 is an explanatory view of (drone) Simplified simulator that applies Example 1.

Since it is also a method to realize a keyboard input method (air keyboard input method) that does not require a keyboard without using an unfinished dedicated device, (1) a method that applies termin, and (2) a method that applies laser communication. Compared to (3) a method using a ring with a gravity sensor, (4) a method of reading surface myoelectric potential, (5) a method of reading brain waves and cerebral blood flow (brain machine interface), etc. The hurdle for commercialization of the present invention is low.

And revealing on a monitor (synthesized on the monitor) translucent by allowing check the relative positional relationship via the also possible mapping thumbnailed, between the brain home position and extracerebral home position Correct the deviation, etc., and reduce the occurrence rate of deviation between the intention (will) of the user and the judgment (recognition) of the electronic device side.
One element constituting the present invention is (1) a method applying a theremin, (2) a method applying laser communication, (3) a method using a ring with a gravity sensor, and (4) a surface myoelectric potential. reading method, (5) method for reading the blood flow of the brain waves and brain (brain machine interface), it is possible to lower the incidence of deviation in like.

When the present invention is accepted and commercialized, the right of claim 3 of the present invention is waived to the keyboard manufacturing company which was established before acceptance or publication and actually put the product on the market. .. Thus, it is advanced further keyboard-less by the invention, the keyboard production company can ensure the preparation period for turning.

Be reduction targets internationally plastics erected as artificial carbon dioxide, if commercialized ahead of the present invention, need not be culled.

(1) Replace the real keyboard and mouse in this case with a real musical instrument and a real controller. (2) Replace the broken real keyboard with no apparent problems and the broken real mouse with no apparent problems with a broken real instrument with no apparent problems and a broken real controller with no apparent problems. (3) Replace the fake keyboard and fake mouse with fake musical instruments and fake controllers. In this way, the air instruments sound comes out, game, radio - Control car, robot, construction machinery, air-controller which is in place of the controller to operate and so are we realized.

1 Monitor 2 Camera 3 The intention (will) of the user judged (recognized) by the electronic device ( or something entered that may or may not be misaligned)
4 Something that has been (or always) pressed (or is pressing) the keyboard of a hand, foot, artificial limb, etc., or something that has been ( or is playing) an instrument 5 Real keyboard ( or) Broken real keyboard with no apparent problems)
6 Real mouse ( or broken real mouse that looks fine)
7 A map of something that the camera has ever ( or always) pressed (or is pressing) on the keyboard of a hand, foot, artificial limb, etc., or something that was playing (or playing) an instrument. (Map of reference numeral 4)
8 envisioned familiar until it can be blind-touch and keyboard (or a selected type of keyboard, or recognized real keyboard to AI in the training, or recognized fake keyboard to AI in the training, or intention of the user of the camera has captured (will) and real electrically broken unconnected fake keyboard for assisting the correction of misalignment, or the camera is captured between the electronic apparatus side determination (recognition) keyboard, or mapping 9 envisioned by and mouse (or selected mouse mode of the camera is not connected to real keyboard that capture is), or real-mice recognized AI in the training, or the camera captures A fake mouse to assist in correcting the discrepancy between the user's intention (will) and the judgment (recognition) of the electronic device, or an electrically broken, unconnected real mouse captured by the camera. Or the image of the unconnected real mouse captured by the camera 10 The intention (will) ( or the deviation) of the user judged (recognized) by the undetermined electronic device may or may not have occurred. has been something), flashes to inform the user (or negative-positive reversal, or discoloration, or white, or huge)
11 Fake keyboard drawn on paper, bags, covers, cases, etc. 12 Fake mouse ( or mascotized fake mouse, or stuffed fake) drawn on paper, bags, covers, cases, etc. -Mouse, or mascotized fake mouse, or keychain-ized fake mouse)
13 selected musical instrument of the type (or AI to the recognized real musical instrument in the training, or fake musical instrument that has been recognized by the AI in the training, or the camera is the intention of the user who captured (will) and of electronic equipment side judgment fake instrument for assisting the correction of the misalignment between the (recognition), or real instruments camera electrically broken unconnected that capture, or unconnected real instrument the camera captures) mapping 14 real instrument ( Or a broken real instrument that looks fine)
15 Fake musical instrument ( or real musical instrument recognized by AI during learning, or fake to assist correction of deviation between the intention (will) of the user captured by the camera and the judgment (recognition) of the electronic device side. Musical instrument)
16 Second mapping that can take on the role of trainer or coach 17 Controller 18 Virtual image captured by the drone-equipped camera that appears on the monitor and changes according to the air input (synthesized on the monitor)

The present invention relates to the input and modification of electronic devices such as electronic computers including tablet PCs, information terminals including smart phones, game machines with monitors, etc., and realizes air instruments that produce sound, games, radios , etc. control car, robot, construction machinery, the present invention relates to an air-controller which is in place of the controller to operate and so on.

Realization of air instruments sound comes out, games, radio control car, robots, construction machinery, also to boost the implementation of the air controller to become a place of the controller to operate and so on, keyboard input method (air that does not require a keyboard -Keyboard input method).

Realization of air instruments sound comes out, games, radio control car, robots, construction machinery, also to boost the implementation of the air controller to become a place of the controller to operate and so on, there is an advantage in that.

By the way, what happens if the tone of an instrument is output to the air input? Furthermore, what happens if radio waves are transmitted like a drone controller?
(1) sound comes out air musical instrument, and (2), games, radio control car, robots, construction machinery, air-controller which is in place of the controller to operate and so on, they can be realized.
One embodiment shown in FIG. 5 is one embodiment (piano, organ) for an air musical instrument that produces sound.

(1) Replace the real keyboard and mouse in this case with a real musical instrument and a real controller. (2) Replace the broken real keyboard with no apparent problems and the broken real mouse with no apparent problems with a broken real instrument with no apparent problems and a broken real controller with no apparent problems. (3) Replace the fake keyboard and fake mouse with fake musical instruments and fake controllers. In this way, the air instruments sound comes out, games, radio control car, robots, construction machinery, air-controller which is in place of the controller to operate and so on to achieve.

With the shift to mobile, keyboard-less has progressed, and at the same time, the usability of the voice input method has improved dramatically. However, some people still find the keyboard input method to be easier to use than the voice input method.
To be on the safe side, I haven't discussed which input method is better. As long as people and situations are diverse, it becomes an unsolvable problem. As long as it is, the question of which input method is superior cannot be argued in the first place. For example, "Voice input is sufficient. Recycling costs are high due to the low proportion of metal, there is a need to reduce regulations on plastics due to marine pollution problems, and it is rational to abolish keyboards." There is a claim. However, the reason why the claim is felt is that (1) it is premised that there are no people who have difficulty speaking , and (2) "it is true that no reasonable accommodation is given." If you are believed to be reasonable it is ", that someone, or limited in both. Of course, there are people who assume that they are different from such reality, people who think that "not reasonable accommodation is true reason", people who do not, and so on. It is also a reality.

Claims (4)

Correct the discrepancy between the user's intention (will) and the judgment (recognition) on the electronic device side through the translucent and thumbnailable mapping that appears on the monitor (combined on the monitor). A keyboard that does not require a keyboard to support the realization of an air instrument that produces sound (increasing the synchronization rate), and the realization of an air controller that replaces the controller that operates games, radio controllers, robots, construction machines, etc. Input method (air keyboard input method, a method that enables keyboard input without a keyboard).
That is, the final communication tool for the user's electronic device such as the hand, foot, artificial hand, etc. recognized by the electronic device synthesized and displayed on the monitor through the camera, and the air keyboard (=) selected by the user. The relative positional relationship between the keyboard that the user called in the electronic device and the keyboard that he intends to use = the completely transparent keyboard = the blind touch that he envisions, the keyboard that he is familiar with until he can touch it = the keyboard in his brain). By making it possible to confirm, it is possible to correct the deviation between the in-brain home position and the out-of-brain home position. For example, for the return key, the first return key is confirmed and the second return key is determined. , It is set as the initial setting, and as a result, the occurrence rate of the deviation between the intention (will) of the user and the judgment (recognition) of the electronic device side is reduced. A broken real keyboard that has no apparent problem, a broken real mouse that has no apparent problem, a paper, a bag, a cover, etc. Fake keyboards drawn on cases, etc., fake mice (or mascotized fake mice, or stuffed fake mice, or mascots) drawn on paper, bags, covers, cases, etc. Keyboard input method that does not require a quasi-keyboard (quasi-air keyboard input method, keyboard input is possible without a quasi-keyboard) that inputs by operating a fake mouse that has been made or a fake mouse that has been made into a key holder. How to do it). A fake drawn on paper, bags, covers, cases, etc. that establishes a keyboard input method (air keyboard input method) that does not require a keyboard and has a function to reduce misjudgment (misrecognition) on the electronic device side. Fake mouse (or mascotized fake mouse, or stuffed fake mouse, or mascotized fake mouse, or key holder) drawn on a keyboard, paper, bag, cover, case, etc. Fake mouse).
However, when the present invention is accepted and commercialized, the right of claim 3 of the present invention shall be granted to the keyboard manufacturing company which was established before acceptance or publication and actually put the product on the market. Abandon. A method to realize an air controller that replaces a controller that operates sound-producing air musical instruments, games, radio controllers, robots, construction machines, etc., which also enables the realization of a simple simulator.
The real keyboard and the real mouse of the first claim, the second claim, and the third claim are replaced with a real musical instrument and a real controller. Replace a broken real keyboard with no apparent problems, a broken real mouse with no apparent problems with a broken real instrument with no apparent problems, or a broken real controller with no apparent problems. Replace fake keyboard and fake mouse with fake instruments and fake controllers. By doing this, you can realize an air controller that replaces the controller that operates sounding air instruments, games, radio-controlled models, robots, construction machinery, and so on.

Images