WO2020203425A1 - 情報処理装置、情報処理方法、及びプログラム - Google Patents

情報処理装置、情報処理方法、及びプログラム Download PDF

Info

Publication number
WO2020203425A1
WO2020203425A1 PCT/JP2020/012787 JP2020012787W WO2020203425A1 WO 2020203425 A1 WO2020203425 A1 WO 2020203425A1 JP 2020012787 W JP2020012787 W JP 2020012787W WO 2020203425 A1 WO2020203425 A1 WO 2020203425A1
Authority
WO
WIPO (PCT)
Prior art keywords
control unit
output
notification sound
information processing
operating body
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/012787
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
高橋　慧
惇一 清水
潤一 永原
学 藤木
知浩 井村
恵一 北原
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Interactive Entertainment Inc
Sony Corp
Original Assignee
Sony Interactive Entertainment Inc
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Interactive Entertainment Inc, Sony Corp filed Critical Sony Interactive Entertainment Inc
Priority to US17/598,075 priority Critical patent/US11947869B2/en
Priority to JP2021511484A priority patent/JPWO2020203425A1/ja
Publication of WO2020203425A1 publication Critical patent/WO2020203425A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R1/00—Details of transducers, loudspeakers or microphones
    • H04R1/10—Earpieces; Attachments therefor ; Earphones; Monophonic headphones
    • H04R1/1041—Mechanical or electronic switches, or control elements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of two-dimensional [2D] relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
    • G06F3/03547—Touch pads, in which fingers can move on a surface
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/165—Management of the audio stream, e.g. setting of volume, audio stream path
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R1/00—Details of transducers, loudspeakers or microphones
    • H04R1/10—Earpieces; Attachments therefor ; Earphones; Monophonic headphones
    • H04R1/1008—Earpieces of the supra-aural or circum-aural type
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R1/00—Details of transducers, loudspeakers or microphones
    • H04R1/10—Earpieces; Attachments therefor ; Earphones; Monophonic headphones
    • H04R1/1016—Earpieces of the intra-aural type
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R2430/00—Signal processing covered by H04R, not provided for in its groups
    • H04R2430/01—Aspects of volume control, not necessarily automatic, in sound systems

Definitions

  • This disclosure relates to information processing devices, information processing methods, and programs.
  • Patent Document 1 discloses a technique for controlling an operation on a device and a sound output that reminds a user of the state of the device.
  • control for dynamically controlling the output of notification information related to a function corresponding to the gesture based on the recognition status of an operating body performing a gesture related to the function execution of the device in a predetermined operation area is provided.
  • the processor operates the output of the notification information related to the function corresponding to the gesture based on the recognition status of the operating body performing the gesture related to the function execution of the device in the predetermined operation area.
  • Information processing methods including gesturing, are provided.
  • the computer outputs notification information related to the function corresponding to the gesture based on the recognition status of the operating body performing the gesture related to the function execution of the device in the predetermined operation area.
  • a program for functioning as an information processing device which includes a control unit for controlling the subject, is provided.
  • a technique for executing a corresponding function based on a predetermined gesture executed by a user in a real space where the proximity sensor can be detected has also been developed.
  • an earphone-type device that can perform processing such as volume adjustment based on a change in the distance between the terminal and the user's hand in real space and is difficult to directly perform a touch operation.
  • processing such as volume adjustment based on a change in the distance between the terminal and the user's hand in real space and is difficult to directly perform a touch operation.
  • a simpler operation that does not require other equipment is realized.
  • the operation on the headphones or earphones worn by the user can be executed in a place that the user cannot see, such as the side surface (housing surface) of the headphones. Therefore, the user cannot confirm whether or not the running gesture can be executed as intended.
  • the terminal recognizes a gesture of a type different from the gesture intended by the user, a situation arises in which the user notices that a different gesture is recognized when a function different from the desired function is executed. obtain.
  • the user cannot grasp in which of the operable areas his / her hand is located in the invisible area. Therefore, a situation may occur in which the user cannot perform the intended operation. For example, when a user swipes in a predetermined direction on the surface of a housing having a touch panel in headphones, if the swipe operation is started from the edge of the surface of the housing, a situation may occur in which a sufficient distance cannot be operated. Further, when the operation is performed based on the gesture in the space, since the user U does not touch the terminal, it may be more difficult to grasp the position in the space where the gesture is performed.
  • the technical idea related to the present disclosure was conceived in view of the above points, and it is possible to confirm the success or failure of the operation and perform a wider variety of operations in an area that cannot be visually recognized by the user.
  • FIG. 1 is a diagram for explaining an outline of the information processing terminal according to the present disclosure.
  • FIG. 1 shows an information processing terminal 10 which is a headphone attached to the user U.
  • the information processing terminal 10 When the information processing terminal 10 recognizes a predetermined gesture operation in an area where the user U, which is an operating body, can operate the housing surface, the information processing terminal 10 executes a function corresponding to the recognized operation.
  • the information processing terminal 10 can notify the information related to the function corresponding to the gesture operation based on the recognition status of the operating body performing the predetermined gesture operation.
  • the notification of the information can be executed by the output of sound.
  • the information processing terminal 10 can recognize at which position in the operation area the hand H of the user U exists, and can output information corresponding to the operation area in which the hand H of the user U is located. is there.
  • the output of the information can be executed by the output of sound.
  • the information processing terminal 10 is a headphone or an earphone will be described.
  • FIG. 2 is a diagram for explaining an example of the functional configuration of the information processing terminal 10 according to the present embodiment.
  • the information processing terminal 10 includes an input unit 110, a recognition unit 120, a control unit 130, an output unit 140, a storage unit 150, and a communication unit 160.
  • the input unit 110 receives various information.
  • the input unit 110 receives, for example, an operation input from a user.
  • the input unit 110 may be provided with a touch pad on the housing surface, for example, and may accept a touch operation from the user.
  • the input unit 110 may detect the contact of the operating body with the touch pad.
  • the input unit 110 may include a proximity sensor to detect an operating body in real space.
  • the area used for the operation among the areas where the input unit 110 can detect the operation body is also referred to as an operation area.
  • the input unit 110 may be provided with a microphone for capturing external sound.
  • the external sound signal captured by the input unit 110 through the microphone can be used for the external sound removing process.
  • the information received by the input unit 110 is transmitted to the recognition unit 120.
  • the recognition unit 120 executes various recognition processes based on the information transmitted from the input unit 110.
  • the recognition unit 120 recognizes that a predetermined operation is being executed.
  • the recognition unit 120 has a gesture recognition function. Based on the detection results of the touchpad and the proximity sensor, the recognition unit 120 can recognize the slide on the touchpad and the execution of the gesture in the real space that can be detected by the proximity sensor.
  • the recognition result by the recognition unit 120 is transmitted to the control unit 130 as a recognition status.
  • Control unit 130 The control unit 130 controls each configuration of the information processing terminal 10.
  • control unit 130 may dynamically control the output of the notification information related to the function corresponding to the gesture based on the recognition status of the operating body performing the gesture in the operation area.
  • the recognition status of the operating body is the recognition result recognized by the recognition unit 120.
  • the control unit 130 controls the output of the broadcast information according to various situations including the recognition status.
  • the notification information is information related to the gesture being executed.
  • the information related to the gesture being executed is, for example, information that allows the user U to understand the type of gesture recognized by the recognition unit 120.
  • the notification information is, for example, a notification sound.
  • the notification information is a notification sound
  • the notification information may be expressed by other than the sound information.
  • Examples of the broadcast information include vibration information and the like.
  • the control unit 130 may control the output of the notification information related to the operation area in which the operation body is located, based on the position of the operation body indicated by the recognition status.
  • the notification information related to the operation area includes, for example, information indicating whether or not the operation body is located in the operation area, or when a plurality of operation areas exist, the operation body is located in which operation area. Information indicating that.
  • the notification information related to the operation area is the notification sound related to the operation area will be described.
  • the control unit 130 controls the execution of the function corresponding to the operation area in which the operating body is located.
  • the functions include, for example, an output volume adjusting function, a song forward function, and a song return function.
  • the output unit 140 outputs various information under the control of the control unit 130.
  • the output unit 140 outputs sound by, for example, a driver unit. Examples of the sound output by the output unit 140 include the above-mentioned notification sound. Further, the output unit 140 executes a predetermined function under the control of the control unit 130. Further, the output unit 140 may include an actuator and output notification information by vibration.
  • the storage unit 150 stores various information related to the operation of the information processing terminal 10.
  • the storage unit 150 may store, for example, sound source information used as a notification sound.
  • the communication unit 160 performs information communication with other devices. For example, the communication unit 160 receives sound information from another device. The sound information received by the communication unit 160 is output to the user U by the output unit 140 via, for example, the control unit 130. Other devices include, for example, music players and smartphones.
  • the control unit 130 may dynamically control the output of the notification sound based on the type of gesture being executed by the operating body indicated by the recognition status.
  • the type of gesture means, for example, a type determined based on how the operating body moves in the operation area.
  • the types of gestures include horizontal sliding movement of the operating body on the touchpad and rotational movement in a circular motion.
  • FIG. 3 is a diagram for explaining an example of output control of a notification sound based on the type of gesture by the control unit 130 according to the present embodiment.
  • FIG. 3 shows an information processing terminal 10 which is a headphone attached to the user U.
  • the hand H of the user U which is the operating body, is moving horizontally (flicking) on the touch pad.
  • the control unit 130 indicates that the hand H of the user U is moving in the horizontal direction depending on the recognition status, and causes the output unit 140 to output a notification sound corresponding to the movement in the horizontal direction. There is.
  • the hand H of the user U is moving so as to rotate on the touch pad.
  • the control unit 130 indicates that the hand H of the user U is rotating and moving depending on the recognition status, and outputs a notification sound corresponding to the movement in the horizontal direction to the output unit 140.
  • the notification sound output by the output unit 140 is different from the notification sound output on the left side of FIG.
  • the notification sound is output based on the type of the gesture being executed, but the control unit 130 also has the number of fingers of the hand H of the user U who is executing the gesture.
  • the output of the notification sound may be controlled based on the above. As a result, even when the same gesture is being executed, the notification sound corresponding to the number of each finger can be output, and it is confirmed whether or not the gesture is recognized by the number of fingers intended by the user U. It becomes possible to do.
  • control unit 130 may output a notification sound based on the moving speed of the operating body that is executing the gesture. As a result, it is possible to check the degree of change of the set value and execute a more detailed operation. Further, when the type of the gesture being executed cannot be specified, the control unit 130 outputs a notification sound corresponding to the preliminary operation of the gesture until the type can be specified, and is executing at the stage when the type can be specified. A notification sound may be output based on the type of gesture.
  • FIG. 4 is a diagram for explaining an example of output control of a notification sound based on the number of fingers of the user by the control unit 130 according to the present embodiment.
  • FIG. 4 shows an information processing terminal 10 which is a headphone attached to the user U.
  • one finger of the user U's hand H is moving horizontally (flicking) on the touch pad.
  • the control unit 130 indicates that the gesture is being executed with two fingers depending on the recognition status
  • the output unit 140 outputs a notification sound corresponding to the movement in the horizontal direction.
  • the output unit 140 outputs a notification sound corresponding to the movement in the horizontal direction.
  • the notification sound output by the output unit 140 is different from the notification sound output on the left side of FIG.
  • the output notification sound changes based on the type of gesture being executed and the number of fingers performing the gesture.
  • the control unit 130 may output the notification sound only when the number of fingers of the user U as the operating body is a predetermined number depending on the recognition status. For example, in the case of the example shown in FIG. 4, the control unit 130 continues to touch the touch pad with, for example, two fingers only when the recognition status indicates that the gesture is being performed with two fingers. The notification sound may be output only during the period. As a result, the user U can more clearly grasp the number of recognized fingers.
  • the control unit 130 may dynamically control the output of the notification sound based on the moving direction of the operating body indicated by the recognition status. Specifically, the control unit 130 may change the output mode of the notification sound based on the movement direction of the operating body indicated by the recognition status.
  • the output mode of the notification sound here is, for example, the frequency, pitch, density, volume, etc. of the notification sound.
  • FIGS. 5 to 7 are diagrams for explaining an example of output control of the notification sound based on the moving direction of the operating body by the control unit 130 according to the present embodiment.
  • FIG. 5 shows a graph G1 showing a change in the notification sound based on the moving direction of the operating body.
  • the pitch of the notification sound may gradually increase as the operating body moves to the right, while the pitch of the notification sound may gradually decrease as the operating body moves to the left.
  • the density of the sound output as the notification sound gradually increases as the operating body moves upward, while the density of the sound is gradually output as the notification sound as the operating body moves downward.
  • the density of the sound produced may decrease.
  • the pitch and density of the notification sound may be determined based on the position of the operating body in the area PR.
  • the control unit 130 may output a notification sound following the movement of the operating body.
  • FIG. 6 and 7 show an information processing terminal 10 which is a headphone attached to the user U.
  • the control unit 130 makes a notification sound. Gradually raise the pitch.
  • the control unit 130 gradually increases the density of the notification sound when the recognition unit 120 recognizes that the operating body is moving from the lower side to the upper side of the touch pad. ..
  • the frequency and volume of the notification sound may be determined based on the moving direction of the operating body.
  • the pitch and density of the notification sound may be determined based on the moving direction of the operating body.
  • FIG. 8 is a diagram for explaining an example of changes in the pitch and density of the notification sound by a single sound source according to the present embodiment.
  • FIG. 8 shows graphs G2 to G4.
  • Graph G2 repeatedly shows a single sound source.
  • a single sound source having a higher density than the graph G2 is repeatedly shown.
  • the pitch of a single sound source is reduced as compared with the graphs G2 and G3.
  • the control unit 130 may dynamically control the output of the notification sound based on the comparison between the moving distance of the operating body and the predetermined threshold value. Specifically, the control unit 130 may dynamically control the output of the notification sound based on the difference between the moving distance of the operating body and a predetermined threshold value.
  • the moving distance of the operating body is the moving distance of the operating body in the operating area
  • the predetermined threshold value may be a predetermined value.
  • FIG. 9 is a diagram for explaining an example of output control of a notification sound based on a comparison between a moving distance of an operating body and a predetermined threshold value by the control unit 130 according to the present embodiment.
  • FIG. 9 shows a graph G5 showing that the operating body has moved by a predetermined distance or more, and a graph G6 showing the pitch of the notification sound corresponding to the graph G5.
  • FIG. 9 shows a graph G7 showing that the operating body stopped moving before moving by a predetermined distance or more, and a graph G8 showing the pitch of the notification sound corresponding to the graph G7.
  • the horizontal axis of the graphs G5 and G7 is the time, and the vertical axis is the position based on the initial position of the operating body.
  • the horizontal axis of the graphs G6 and G8 is the time, and the vertical axis is the pitch of the notification sound.
  • the pitch of the notification sound gradually increases as the operating body moves, and the pitch of the notification sound may also increase even after the movement distance exceeds a predetermined threshold value.
  • the control unit 130 may execute the function related to the information processing terminal 10 when the moving distance of the operating body exceeds a predetermined threshold value.
  • the pitch of the notification sound gradually increases as the operating body moves, but the operation is completed before the moving distance exceeds a predetermined threshold value.
  • the change in the pitch of the notification sound may change from an increase to a decrease.
  • the pitch of the notification sound may be substantially the same as the pitch of the notification sound at the moment when the operating body starts the operation. Note that the control unit 130 does not perform the same function because the moving distance of the operating body does not exceed a predetermined threshold value.
  • the way of changing the pitch of the notification sound differs depending on whether or not the moving distance of the operating body exceeds a predetermined threshold value. Therefore, the user U can intuitively grasp whether or not the corresponding function can be executed.
  • the output control of the notification sound described with reference to FIG. 9 is merely an example, and is not limited to such an example.
  • control unit 130 may control the sound output related to the operation area in which the operation body is located, based on the position of the operation body indicated by the recognition status. That is, the control unit 130 may give feedback to the user U regarding the position of the operating body based on the position of the operating body indicated by the recognition status. For example, when there are a plurality of operation areas, the control unit 130 may control the sound output based on which operation area the operating body is located in.
  • FIG. 10 is a diagram for explaining an example of the operation area according to the present embodiment.
  • FIG. 10 shows an operable area OR and an inoperable area NOR on the touch pad of the information processing terminal 10 and the information processing terminal 10 which are headphones attached to the user U.
  • the operable area OR is a circular operating area that can be operated by the operating body and cannot be visually recognized by the user U.
  • the control unit 130 relates to an information processing terminal 10 corresponding to the operable area OR. You may control the performance of the function.
  • the inoperable region NOR is an region that cannot be operated by the operating body.
  • the arrangement and shape of the operable area OR and the inoperable area NOR are not limited to such an example.
  • FIG. 11 is a diagram for explaining an example of output control of the notification sound corresponding to the operation area where the operation body is located by the control unit 130 according to the present embodiment.
  • FIG. 11 shows an information processing terminal 10 which is a headphone and operation areas R1 to R4 attached to the user U.
  • the operation areas R1 to R4 are the same areas as the operable area OR shown in FIG.
  • the hand H of the user U On the left side of FIG. 11, the hand H of the user U, which is the operating body, is located in the operating area R1.
  • the control unit 130 indicates that the hand H of the user U is located in the operation area R1 depending on the recognition status, and outputs the notification sound corresponding to the operation area R1 to the output unit 140.
  • the hand H of the user U On the right side of FIG. 11, the hand H of the user U, which is the operating body, is located in the operating area R2.
  • the control unit 130 indicates that the hand H of the user U is located in the operation area R2 depending on the recognition status, and causes the output unit 140 to output the notification sound corresponding to the operation area R2.
  • the notification sound output by the output unit 140 is different from the notification sound output on the left side of FIG. In this way, when there are a plurality of operation areas, the user U can grasp which operation area the operation body is located in without visually recognizing it.
  • control unit 130 may output a notification sound for notifying the expected movement to the outside of the operating area.
  • FIG. 12 is a diagram for explaining an example of output control of the notification sound when the operating body by the control unit 130 according to the present embodiment is expected to move out of the operation area.
  • FIG. 12 shows an information processing terminal 10 which is a headphone and operation areas R1 to R4 attached to the user U.
  • the control unit 130 causes the output unit 140 to output a notification sound corresponding to the operation area R1.
  • the hand H of the user U which is the operating body, is located in the operating area R2, but is in a state of moving outward from the operating area R2. In such a state, the control unit 130 outputs a notification sound for notifying the prediction that the operating body will move out of the operating area R2.
  • the notification sound output on the right side of FIG. 12 may be a sound that warns that movement is expected outside the operation area.
  • the user U can correct the position of the operating body even when the operating body is about to move out of the operating area. It should be noted that it may be considered that the operating body is expected to move out of the operating area when it is located at the end in the operating area without considering the moving direction of the operating body.
  • the control unit 130 may control the output of the notification sound related to the plurality of operation areas. Specifically, the control unit 130 may output a notification sound corresponding to each operation area based on the positional relationship between the operation body and each operation area. For example, the control unit 130 may determine the volume, pitch, frequency, etc. when outputting the notification sound corresponding to each operation area based on the positional relationship between the operation body and each operation area. The volume, pitch, frequency, and the like of the notification sound may be determined based on, for example, the distance from each operation area. Further, the control unit 130 may control the output of the notification sound so as to guide the user to a predetermined operation area.
  • FIG. 13 is a diagram for explaining an example of output control of the notification sound corresponding to each operation area by the control unit 130 according to the present embodiment.
  • FIG. 13 shows an information processing terminal 10 which is a headphone and an operable area OR1 and OR2 which are attached to the user U.
  • the control unit 130 outputs so that the closer the distance between the operating body and each of the operable areas OR1 and OR2 is, the higher the volume of the notification sound corresponding to each of the operable areas OR1 and OR2 is.
  • the position where the user U's hand H is in contact with the touch pad is a position closer to the operable area OR2 than to the operable area OR1. Therefore, the control unit 130 outputs the notification sound corresponding to the operable area OR2 at a volume louder than that of the operable area OR1.
  • the user U can determine the relative positional relationship between the operation body and the plurality of operation areas. It becomes possible to grasp.
  • the control unit 130 is based on the distance between the operation body and each operation area in the real space.
  • the notification sound corresponding to the operation area may be output.
  • FIG. 14 is a diagram for explaining an example of output volume control of the notification sound corresponding to each operation area by the control unit 130 according to the present embodiment.
  • FIG. 14 shows a graph BG in which the horizontal axis is the position and the vertical axis is the volume, and the boundary points LP and RP between the operation area and other areas are present.
  • the boundary points LP and RP are boundary points with different operation areas, and the boundary points LP and RP are not operation areas.
  • the intermediate point M is a point where the distances between the boundary points LP and RP are equal.
  • the graph BG shows the volume LB of the notification sound corresponding to the boundary point LP and the volume RB of the notification sound corresponding to the boundary point RP when the operating body is located between the boundary point LP and the RP. ..
  • the control unit 130 outputs a notification sound so that the closer the position of the operating body is to each boundary point, the louder the volume.
  • the total of the volume LB of the notification sound corresponding to the boundary point LP and the volume RB of the notification sound corresponding to the boundary point RP is constant so that the user U can easily recognize the change in the volume of each notification sound. There is.
  • the volume of each notification sound is half of the maximum value.
  • the user U can grasp the position of the operating body more intuitively.
  • the control unit 130 may change the information notified by the notification sound based on the positional relationship between the operating body and the group of the plurality of operating areas. For example, when the operating body is located between the plurality of operating areas, the control unit 130 may output a notification sound as shown in FIG. On the other hand, for example, when the operating body is located in an area outside the area where the plurality of operating areas exist, it may be prioritized that the hand H of the user U moves to the area where the plurality of operating areas exist. The 130 may output a notification sound notifying that it is located outside the plurality of operation areas.
  • FIG. 15 is a diagram for explaining an example of the positional relationship between the operating body and the plurality of operating regions according to the present embodiment.
  • FIG. 15 shows the user U's hand H and the operable areas OR1 and OR2.
  • each operation of the control unit 130 is based on the positional relationship between the operating body and each operating area.
  • the notification sound corresponding to the area is output.
  • the control unit 130 outputs only the notification sound corresponding to the operable area OR1.
  • the notification sound output at the position of the operating body shown on the left side of FIG. 15 is different from the notification sound. It outputs a notification sound notifying that it is located outside the different operable areas OR1 and OR2.
  • the user U can operate the terminal more comfortably.
  • control unit 130 may notify the user U by sound only when the operating body is located outside the operable areas OR1 and OR2. This makes it possible for the user U to reliably warn when the operating body is significantly out of the plurality of operating areas.
  • the notification can be applied as a more reliable warning even when the information processing terminal 10 is an HMD (Head Mounted Display).
  • the control unit 130 may control the output of the notification sound corresponding to the reference region.
  • FIG. 16 is a diagram for explaining an example of output control of the notification sound related to the reference region by the control unit 130 according to the present embodiment.
  • FIG. 16 shows an information processing terminal 10 which is a headphone and a reference area SR attached to the user U.
  • the reference region SR is a predetermined region and is a region suitable as a start region of the operation.
  • the hand H of the user U which is the operating body, is moving horizontally (flicking) on the touch pad.
  • the hand H of the user U is located outside the reference region SR.
  • the notification sound is not output.
  • the hand H of the user U is located in the reference region SR.
  • the control unit 130 indicates that the hand H of the user U is located in the reference region SR depending on the recognition status, and outputs a notification sound corresponding to the reference region SR.
  • control unit 130 outputs the notification sound corresponding to the reference area SR, so that the operation can be performed from the reference point, and more accurate operation becomes possible.
  • the control unit 130 did not output a notification sound when the operating body was not located in the reference area SR, but on the other hand, the control unit 130 notified to guide to the reference area SR. Sound may be output.
  • the control unit 130 notified to guide to the reference area SR. Sound may be output.
  • the operating body performs the sliding operation in the predetermined direction once, then returns to the reference region SR, and then performs the sliding operation in the same direction again.
  • the guidance to the reference region SR by the notification sound is realized by, for example, the method shown in FIGS. 13 to 15. Further, the reference region SR is not limited to the size shown in FIG.
  • the reference region SR was present at a predetermined location.
  • the reference area SR does not always make the operation easier. Therefore, the existing location may be determined by the user U.
  • the place where the user U first starts the operation in the operation area may be determined as the reference area SR.
  • the "location where the operation is first started” means, for example, the location where the operating body for executing one function is first located on the operation area.
  • the “location where the operation is first started” may be, for example, a location where the operating body is first located on the operation area when performing a plurality of operations in succession.
  • the definition of "where the operation first starts” is not limited to such an example.
  • FIG. 17 is a diagram for explaining an example of output control of a notification sound related to a reference region whose position is arbitrarily determined by the control unit 130 according to the present embodiment.
  • FIG. 17 shows an information processing terminal 10 which is a headphone attached to the user U.
  • the control unit 130 determines the portion first touched by the hand H of the user U as the reference region SR.
  • the reference region SR determined by the control unit 130 is shown on the right side of FIG. 17, the reference region SR determined by the control unit 130 is shown.
  • the control unit 130 since the control unit 130 indicates that the hand H of the user U is located in the reference region SR depending on the recognition status, the control unit 130 outputs a notification sound corresponding to the reference region SR.
  • the information processing terminal 10 may be earphones.
  • the earphone which is the information processing terminal 10 may be provided with a touch sensor as an input unit 110, or may be provided with a proximity sensor. Since the earphones are smaller than the headphones, it may be difficult to operate them with the touch pad, and it may be suitable to operate them in space with the proximity sensor.
  • the earphone has a proximity sensor as the input unit 110 will be described. The proximity sensor recognizes the position of the operating body in space.
  • FIG. 18 is a diagram for explaining an example in the case where the operation area according to the present embodiment exists in the space.
  • FIG. 18 shows an information processing terminal 10 which is an earphone and operation areas AR1 to AR4 in space.
  • the control unit 130 may control the output of the notification sound when the operating body is located in any of the operation areas AR1 to AR4. Further, the control unit 130 may dynamically control the output of the notification information related to the function corresponding to the gesture based on the recognition status of the operating body performing the gesture in the operation areas AR1 to AR4.
  • control unit 130 may control the output of the notification sound based on the distance.
  • FIG. 19 describes an example of execution control of the function related to the information processing terminal 10 based on the distance when the distance between the operating body and the information processing terminal 10 is shown as the recognition status by the control unit 130 according to the present embodiment. It is a figure for.
  • FIG. 19 shows the information processing terminal 10 which is an earphone and the hand H of the user U.
  • the hand H of the user U and the information processing terminal 10 are separated by a distance D3.
  • the control unit 130 processes the user U's hand H according to the recognition status.
  • the output of the notification sound based on the change in the distance from the distance D3 to the distance D4 is controlled.
  • the volume of the notification sound output by the information processing terminal 10 is increased. It's getting bigger.
  • control unit 130 may control the output of the notification sound based on whether or not the operating body exists in the operation area. Specifically, the control unit 130 may control the output of the notification sound based on whether or not the operating body exists in the space where the input unit 110 can detect.
  • FIG. 20 shows an information processing terminal 10 which is an earphone and operation areas AR1 to AR4 in space.
  • the hand H of the user U which is the operating body, exists outside the space of the operating area. Since it is shown that the hand H of the user U exists outside the space of the operation area, the control unit 130 does not output the notification sound.
  • the hand H of the user U is located in the space of the operation area.
  • the control unit 130 indicates that the hand H of the user U is located in the space of the operation area depending on the recognition status, and outputs a notification sound corresponding to the reference area SR.
  • the specific example of the output control of the notification sound by the control unit 130 has been described above.
  • the example in which the information processing terminal 10 is a headphone or an earphone has been described, but the present invention is not limited to this example.
  • the information processing terminal 10 may be, for example, an in-vehicle touch panel. It is difficult to visually recognize the touch panel for performing various operations while driving a car. At that time, the information processing terminal 10 provided on the back of the steering wheel notifies the information by a notification sound or vibration, so that various operations can be executed.
  • the information processing terminal 10 may be an autonomous mobile body or the like. While the user U is performing a gesture operation on the autonomous moving body, the autonomous moving body appropriately gives feedback by a notification sound or vibration, so that the user U can enjoy the interaction with the autonomous moving body more.
  • the above technology can also be applied to a device that provides VR (Virtual Reality) content.
  • the user U is often in a situation where he / she cannot visually recognize the controller or the like while using the VR content. Therefore, the user U can perform a more accurate operation while listening to the notification sound regarding the operation of the controller.
  • the input unit 110 detects the operating body, and the recognition unit 120 recognizes the position of the detected operating body (S101).
  • the control unit 130 changes the pitch and frequency of the notification sound according to the moving distance of the operating body based on the position of the operating body recognized in step S101 (S102).
  • step S101 when the moving distance of the operating body based on the position of the operating body recognized in step S101 exceeds a predetermined threshold value (S103: YES), the control unit 130 raises the frequency of the notification sound (S104) to provide information.
  • the processing terminal 10 ends the operation.
  • the moving distance of the operating body based on the position of the operating body recognized in step S101 does not exceed a predetermined threshold value (S103: NO) and the operation by the operating body is not completed (S105: NO). Return to step S101.
  • step S101 when the moving distance of the operating body based on the recognized position of the operating body does not exceed a predetermined threshold value (S103: NO) and the operation by the operating body is completed (S105: YES).
  • the control unit 130 reduces the frequency of the notification sound (S106), and the information processing terminal 10 ends the operation.
  • control unit 130 executes a pre-process in the recognition process of the operating body (S201).
  • the control unit 130 starts accepting gestures (S203).
  • step S204 when the preparatory movement of the gesture is not recognized by the recognition unit 120 (S204: NO), the process returns to step S202.
  • the recognition unit 120 recognizes the preparatory movement of the gesture (S204: YES) and recognizes that the gesture is continuously executed after the preparatory movement of the gesture (S205: YES)
  • the control unit 130 Controls the output of the notification sound recognized in step S205 (S206), and the information processing terminal 10 ends the operation.
  • the recognition unit 120 recognizes the preparatory movement of the gesture (S204: YES) and does not recognize that the gesture is continuously executed after the preparatory movement of the gesture (S205: NO), the control unit 130 The acceptance of the gesture is finished (S207), and the information processing terminal 10 ends the operation.
  • the control unit 130 notifies according to the distance of the operating body to each operating area.
  • the pitch and frequency of the sound are changed (S304), and the control unit 130 ends the operation.
  • the control unit 130 responds to the distance of the operating body to one operating area.
  • the pitch and frequency of the notification sound are changed (S305), and the control unit 130 ends the operation.
  • FIG. 24 is a block diagram showing a hardware configuration example of the information processing terminal 10 according to the embodiment of the present disclosure.
  • the information processing terminal 10 includes, for example, a processor 871, a ROM 872, a RAM 873, a host bus 874, a bridge 875, an external bus 876, an interface 877, an input device 878, and an output device 879. It has a storage 880, a drive 881, a connection port 882, and a communication device 883.
  • the hardware configuration shown here is an example, and some of the components may be omitted. Further, components other than the components shown here may be further included.
  • the processor 871 functions as, for example, an arithmetic processing unit or a control device, and controls all or a part of the operation of each component based on various programs recorded in the ROM 872, the RAM 873, the storage 880, or the removable recording medium 901. ..
  • the ROM 872 is a means for storing a program read into the processor 871 and data used for calculation.
  • a program read into the processor 871 and various parameters that change as appropriate when the program is executed are temporarily or permanently stored.
  • the processors 871, ROM 872, and RAM 873 are connected to each other via, for example, a host bus 874 capable of high-speed data transmission.
  • the host bus 874 is connected to the external bus 876, which has a relatively low data transmission speed, via, for example, the bridge 875.
  • the external bus 876 is connected to various components via the interface 877.
  • Input device 8708 For the input device 878, for example, a mouse, a keyboard, a touch panel, buttons, switches, levers, and the like are used. Further, as the input device 878, a remote controller (hereinafter, remote controller) capable of transmitting a control signal using infrared rays or other radio waves may be used. Further, the input device 878 includes a voice input device such as a microphone.
  • the output device 879 provides the user with acquired information such as a display device such as a CRT (Cathode Ray Tube), an LCD, or an organic EL, an audio output device such as a speaker or headphones, a printer, a mobile phone, or a facsimile. It is a device that can notify visually or audibly. Further, the output device 879 according to the present disclosure includes various vibration devices capable of outputting tactile stimuli.
  • the storage 880 is a device for storing various types of data.
  • a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, an optical magnetic storage device, or the like is used.
  • the drive 881 is a device that reads information recorded on a removable recording medium 901 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, or writes information on the removable recording medium 901.
  • a removable recording medium 901 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory
  • the removable recording medium 901 is, for example, a DVD media, a Blu-ray (registered trademark) media, an HD DVD media, various semiconductor storage media, and the like.
  • the removable recording medium 901 may be, for example, an IC card equipped with a non-contact type IC chip, an electronic device, or the like.
  • connection port 882 is a port for connecting an external connection device 902 such as a USB (Universal Serial Bus) port, an IEEE1394 port, a SCSI (SCSI Computer System Interface), an RS-232C port, or an optical audio terminal. is there.
  • an external connection device 902 such as a USB (Universal Serial Bus) port, an IEEE1394 port, a SCSI (SCSI Computer System Interface), an RS-232C port, or an optical audio terminal.
  • the externally connected device 902 is, for example, a printer, a portable music player, a digital camera, a digital video camera, an IC recorder, or the like.
  • the communication device 883 is a communication device for connecting to a network, and is, for example, a communication card for wired or wireless LAN, Wireless (registered trademark), or WUSB (Wireless USB), a router for optical communication, and ADSL (Asymmetric Digital). A router for Subscriber Line), a modem for various communications, and the like.
  • the information processing terminal 10 can confirm the success or failure of the operation and perform a wider variety of operations in a region that cannot be visually recognized by the user.
  • a control unit that dynamically controls the output of notification information related to the function corresponding to the gesture based on the recognition status of the operating body performing the gesture related to the function execution of the device in the predetermined operation area.
  • the notification information is a notification sound.
  • the control unit dynamically controls the output of the notification sound based on the recognition status.
  • the recognition status is the moving direction of the operating body.
  • the control unit dynamically controls the output of the notification sound based on the moving direction of the operating body indicated by the recognition status.
  • the information processing device according to (2) above.
  • the control unit changes one or both of the frequency and volume of the notification sound based on the movement direction indicated by the recognition status.
  • the information processing device is the type of the gesture being executed by the operating body.
  • the control unit dynamically controls the output of the notification sound based on the type indicated by the recognition status.
  • the information processing device according to any one of (2) to (4) above.
  • the recognition status is a preliminary operation of the gesture.
  • the control unit dynamically controls the output of the notification sound based on the preliminary operation indicated by the recognition status.
  • the information processing device according to (5) above.
  • the control unit dynamically controls the output of the notification sound based on the specified type indicated by the recognition status.
  • the information processing device according to (6) above.
  • the recognition status is the moving speed of the operating body.
  • the control unit dynamically controls the output of the notification sound based on the moving speed indicated by the recognition status.
  • the information processing device according to any one of (2) to (7) above.
  • the operating body is a user's hand.
  • the recognition status is the number of fingers of the user's hand.
  • the control unit dynamically controls the output of the notification sound based on the number of fingers indicated by the recognition status.
  • (11) The control unit dynamically controls the output of the notification sound related to the execution of the function corresponding to the number of fingers indicated by the recognition status.
  • the information processing device according to (10) above.
  • (12) The control unit dynamically controls the output of the notification sound composed of a single sound source based on the recognition status.
  • the information processing device changes one or both of the notification sound pitch and the density of the notification sound composed of the single sound source based on the recognition situation.
  • the information processing device according to (12) above.
  • the recognition status is the moving distance of the operating body.
  • the control unit dynamically controls the output of the notification sound based on the comparison between the moving distance indicated by the recognition status and a predetermined threshold value.
  • the information processing device according to any one of (2) to (13) above.
  • the control unit dynamically controls the output of the notification sound based on the difference between the moving distance and the predetermined threshold value.
  • the information processing device 14 above.
  • the processor Dynamically controlling the output of notification information related to the function corresponding to the gesture based on the recognition status of the operating body performing the gesture related to the function execution of the device in the predetermined operation area. including, Information processing method.
  • Computer A control unit that dynamically controls the output of notification information related to the function corresponding to the gesture based on the recognition status of the operating body performing the gesture related to the function execution of the device in the predetermined operation area.
  • Information processing terminal 110 Input unit 120 Recognition unit 130 Control unit 140 Output unit 150 Storage unit 160 Communication unit

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • User Interface Of Digital Computer (AREA)
PCT/JP2020/012787 2019-04-01 2020-03-23 情報処理装置、情報処理方法、及びプログラム Ceased WO2020203425A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US17/598,075 US11947869B2 (en) 2019-04-01 2020-03-23 Information processing device, information processing method, and program
JP2021511484A JPWO2020203425A1 (https=) 2019-04-01 2020-03-23

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019069643 2019-04-01
JP2019-069643 2019-04-01

Publications (1)

Publication Number Publication Date
WO2020203425A1 true WO2020203425A1 (ja) 2020-10-08

Family

ID=72668855

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/012787 Ceased WO2020203425A1 (ja) 2019-04-01 2020-03-23 情報処理装置、情報処理方法、及びプログラム

Country Status (3)

Country Link
US (1) US11947869B2 (https=)
JP (1) JPWO2020203425A1 (https=)
WO (1) WO2020203425A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12564794B2 (en) 2021-09-10 2026-03-03 Sony Group Corporation Autonomous mobile body and information processing method

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001216069A (ja) * 2000-02-01 2001-08-10 Toshiba Corp 操作入力装置および方向検出方法
JP2012162127A (ja) * 2011-02-04 2012-08-30 Nissan Motor Co Ltd 車載機器用操作装置及び刺激出力方法
WO2012132495A1 (ja) * 2011-03-30 2012-10-04 本田技研工業株式会社 操作装置
JP2017534132A (ja) * 2014-10-10 2017-11-16 ミュージック エルエルシー ユーザー相互作用を共有するための装置

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003131785A (ja) 2001-10-22 2003-05-09 Toshiba Corp インタフェース装置および操作制御方法およびプログラム製品
JP4155831B2 (ja) 2003-01-17 2008-09-24 シャープ株式会社 電気機器の報知装置
JP2007087104A (ja) * 2005-09-22 2007-04-05 Sony Corp 音声出力制御装置および方法、記録媒体、並びに、プログラム
US20100332236A1 (en) * 2009-06-25 2010-12-30 Blueant Wireless Pty Limited Voice-triggered operation of electronic devices
JP2012079082A (ja) * 2010-10-01 2012-04-19 Sony Corp 入力装置
US9360943B2 (en) * 2010-12-27 2016-06-07 Lg Electronics Inc. Display device and method of providing feedback for gestures thereof
US20130339859A1 (en) * 2012-06-15 2013-12-19 Muzik LLC Interactive networked headphones
US10126937B2 (en) * 2016-01-26 2018-11-13 Adobe Systems Incorporated Input techniques for virtual reality headset devices with front touch screens
US10911860B2 (en) * 2016-03-23 2021-02-02 Intel Corporation Automated and body driven headset audio control
CN108370466B (zh) * 2016-04-11 2020-11-10 索尼公司 头戴式耳机、再现控制方法和程序
US10555066B1 (en) * 2017-09-22 2020-02-04 Apple Inc. Detection of headphone rotation
EP3672292B1 (en) * 2018-12-20 2022-10-26 GN Audio A/S Audio communication method and system with a dongle

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001216069A (ja) * 2000-02-01 2001-08-10 Toshiba Corp 操作入力装置および方向検出方法
JP2012162127A (ja) * 2011-02-04 2012-08-30 Nissan Motor Co Ltd 車載機器用操作装置及び刺激出力方法
WO2012132495A1 (ja) * 2011-03-30 2012-10-04 本田技研工業株式会社 操作装置
JP2017534132A (ja) * 2014-10-10 2017-11-16 ミュージック エルエルシー ユーザー相互作用を共有するための装置

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12564794B2 (en) 2021-09-10 2026-03-03 Sony Group Corporation Autonomous mobile body and information processing method

Also Published As

Publication number Publication date
JPWO2020203425A1 (https=) 2020-10-08
US20220179616A1 (en) 2022-06-09
US11947869B2 (en) 2024-04-02

Similar Documents

Publication Publication Date Title
US11388280B2 (en) Device, method, and graphical user interface for battery management
US10425284B2 (en) Device, method, and graphical user interface for establishing a relationship and connection between two devices
US9507507B2 (en) Information processing apparatus, information processing method and program
CN102333266B (zh) 声音信号处理装置、方法、程序以及麦克风装置
WO2022046340A1 (en) Object engagement based on finger manipulation data and untethered inputs
JP7204804B2 (ja) スマートバックミラーのインタラクション方法、装置、電子機器及び記憶媒体
KR20170054423A (ko) 다-표면 컨트롤러
US20150251089A1 (en) Information processing apparatus, information processing system, information processing method, and program
CN205263746U (zh) 一种基于3d手势识别的车载娱乐系统
CN102547511B (zh) 耳机控制方法及耳机
JP2017138738A (ja) 入力装置、表示装置および入力装置の制御方法
US12248636B2 (en) Gesture recognition, adaptation, and management in a head-wearable audio device
WO2020203208A1 (ja) 情報処理装置、情報処理方法、及びプログラム
CN111338594A (zh) 使用指纹的输入控制
WO2020203425A1 (ja) 情報処理装置、情報処理方法、及びプログラム
US20250110571A1 (en) Devices, Methods, and Graphical User Interfaces for Providing Feedback Relating to Real-World Objects
CN121889761A (zh) 用于控制区域的技术
WO2019054037A1 (ja) 情報処理装置、情報処理方法、およびプログラム
JP6433217B2 (ja) 音量制御装置、音量制御システム、及びプログラム
KR102197912B1 (ko) 제스처 인식에 따른 기능 실행 방법, 장치 및 기록매체
CN115136109A (zh) 手势检测系统
JP7536566B2 (ja) オーディオ装置
WO2022060555A1 (en) Merging computer-generated objects based on extremity tracking data
US20230297168A1 (en) Changing a Dimensional Representation of a Content Item
JP2024047795A (ja) 仮想空間インターフェース装置、クライアント端末およびプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20782441

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021511484

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20782441

Country of ref document: EP

Kind code of ref document: A1