WO2024085712A1

WO2024085712A1 - Method and device for producing virtual reality 3d curve network using bimanual interaction

Info

Publication number: WO2024085712A1
Application number: PCT/KR2023/016349
Authority: WO
Inventors: 배석형; 이상현; 진태규; 이준협
Original assignee: 한국과학기술원
Priority date: 2022-10-20
Filing date: 2023-10-20
Publication date: 2024-04-25

Abstract

The present invention relates to a method and device for producing a virtual reality 3D curve network using bimanual interaction, wherein a gesture made by at least one hand of a user is detected in space and a 3D curve network composed of a plurality of lines each defined by two vertexes is produced. The gesture may be based on a pinch gesture made by two fingers and a grab gesture made by five fingers.

Description

Method and device for producing a virtual reality 3D curved network using two-handed interaction

This disclosure relates to a method and device for producing a virtual reality 3D curved network using two-handed interaction.

This patent is research conducted with support from KAIST's large-scale convergence research center operation project.

Virtual reality (VR) allows users to see objects directly at actual size, which is useful for 3D content creation. In such immersive environments, the method of freely drawing 3D curves in the air is widely used due to its excellent expressiveness. However, because this method has low precision, it is difficult to draw precise curves.

This disclosure proposes a new method for generating 3D curves in virtual reality with high expressiveness and precision.

This disclosure provides a method and device for intuitively creating a 3D curved network using two-handed interaction in virtual reality.

According to the present disclosure, a method of creating a virtual reality 3D curved network using two-handed interaction of an electronic device includes detecting a gesture of at least one hand of a user in space, and forming a plurality of lines according to the gesture in the space. and creating a 3D curved network, where each line is defined by two vertices, wherein the gesture is a pinch gesture with two fingers and a pinch gesture with five fingers. It can be based on the grab gesture.

According to the present disclosure, an electronic device for creating a virtual reality 3D curved network using two-handed interaction detects a gesture of at least one hand of a user in space through a camera module, a display module, and the camera module, and displays the display. A processor configured to create, via a module, a 3D curved network in the space in accordance with the gesture, consisting of a plurality of lines, each line defined between two vertices, wherein the gesture is performed using two fingers. It can be based on a pinch gesture and a grab gesture with five fingers.

According to the present disclosure, a user can intuitively create a 3D curved network using two-handed interaction in virtual reality through an electronic device. Specifically, the electronic device allows the user to easily create and modify lines by directly viewing them at actual sizes in virtual reality through gestures with both hands. Therefore, 3D curved networks can be created with high expressiveness and precision in virtual reality.

1 is a block diagram illustrating an electronic device according to various embodiments.

Figure 2 is an example diagram for explaining operation characteristics of an electronic device according to various embodiments.

3 and 4 are conceptual diagrams for explaining gestures used in electronic devices according to various embodiments.

FIGS. 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, and 16 are diagrams of electronic devices according to various embodiments of the present invention. These are example diagrams to explain two-hand interaction based on the gesture of 4.

FIG. 17 is a flowchart illustrating a method of operating an electronic device according to various embodiments.

Hereinafter, various embodiments of the present disclosure are described with reference to the attached drawings.

Hereinafter, the dominant hand (DH) may represent the hand that the user primarily uses among both hands, and the non-dominant hand (NDH) may represent the hand that the user does not primarily use among both hands. . For example, if the user is right-handed, the dominant hand may be the right hand and the non-dominant hand may be the left hand. Meanwhile, if the user is left-handed, the dominant hand may be the left hand, and the non-dominant hand may be the right hand.

FIG. 1 is a block diagram illustrating an electronic device 100 according to various embodiments. FIG. 2 is an example diagram for explaining operation characteristics of the electronic device 100 according to various embodiments.

Referring to FIG. 1, an electronic device 100 may be provided for creating a virtual reality 3D curved network using two-handed interaction. At this time, the electronic device 100 may be implemented as a near-eye display (NED) device that can be worn on the user's face or head. For example, the near-eye display device may include at least one of smart glasses or a head mounted display (HMD) device.

The electronic device 100 includes at least one of a camera module 110, a communication module 120, an input module 320, a display module 140, an audio module 150, a memory 160, or a processor 170. It can be included. In some embodiments, at least one of the components of the electronic device 100 may be omitted, and at least one other component may be added to the electronic device 100. In some embodiments, at least two of the components of the electronic device 100 may be implemented as one integrated circuit.

The camera module 110 can capture images. At this time, when the electronic device 100 is implemented as a near-eye display device, the camera module 110 can capture a front image of the user wearing the electronic device 100. For example, the camera module 110 may include at least one lens, at least one image sensor, at least one image signal processor, and at least one flash.

The communication module 120 may perform communication with an external device (not shown) in the electronic device 100. The communication module 120 may establish a communication channel between the electronic device 100 and an external device and perform communication with the external device through the communication channel. The communication module 120 may include at least one of a wired communication module or a wireless communication module. For example, the wireless communication module may communicate with an external device through at least one of a long-distance communication network or a local area communication network.

The input module 130 may input a signal to be used in at least one component of the electronic device 100. The input module 130 may be configured to detect a signal directly input by the user or to generate a signal by detecting changes in the surroundings. For example, the input module 130 may include at least one of a microphone, at least one physical button, or a touch pad. The touch pad may be configured to detect contact by the user's hand, that is, a touch, and generate a touch signal in response. Here, the touch pad may include at least one of a touch circuitry configured to detect a touch or a sensor circuit configured to measure the intensity of force generated by the touch.

The display module 140 may visually output information to the outside of the electronic device 100. Specifically, the display module 140 may be configured to display visual content. At this time, when the electronic device 100 is implemented as a near-eye display device, the display module 140 may be placed in front of the eyes of the user wearing the electronic device 100. The display module 140 may have a different display method depending on the type of electronic device 100. For example, when the electronic device 100 is an optical see-through type, at least a portion of the display module 140 is made of a transparent or translucent material, and the user can view the actual environment through the display module 140. You can see it for yourself. In this case, as the display module 140 displays the visual content, the user can view the real environment and the visual content together. As another example, if the electronic device 100 is a video see-through type, the user can view images of the actual environment captured through the first camera module 111 through the display module 140. In this case, as the display module 140 displays the visual content, the user can view the image of the real environment and the visual content together.

The audio module 150 can output information audibly to the outside of the electronic device 100. For example, the audio module 150 may include at least one of a speaker or a receiver.

The memory 160 may store various data used by at least one component of the electronic device 100. For example, the memory 160 may include at least one of volatile memory and non-volatile memory. Data may include at least one program and input or output data related thereto. The program may be stored in the memory 160 as software including at least one command, and may include at least one of an operating system, middleware, or an application.

The processor 170 may execute a program in the memory 160 to control at least one component of the electronic device 100. Through this, the processor 170 can process data or perform calculations. At this time, the processor 170 may execute instructions stored in the memory 160.

According to various embodiments, the electronic device 100 can create a virtual reality 3D curved network using the user's two-handed interaction. Specifically, while the user is wearing the electronic device 100, the electronic device 100 may create a virtual 3D curved network according to a gesture of at least one hand of the user in space. At this time, the 3D curved network consists of a plurality of lines, and each line can be defined between two vertices. Here, each line may be a straight line or a curved line. For example, the electronic device 100 can create a 3D curved network for automobile design, as shown in FIG. 2 .

FIGS. 3 and 4 are conceptual diagrams for explaining gestures used in the electronic device 100 according to various embodiments.

Referring to Figure 3, gestures may be based on pinch gestures and grab gestures. A pinch gesture can be formed by two fingers of one hand, and a grab gesture can be formed by five fingers of one hand. Based on this, gestures can include loose-pinch gestures, tight-pinch gestures, loose-grab gestures, and tight-grab gestures. there is.

Specifically, as shown in (a) of FIG. 3, two fingers may be spaced apart from each other in a loose-pinch gesture, and as shown in (b) of FIG. 3, two fingers may be spaced apart from each other in a tight-pinch gesture. The dog's fingers may be pressed together. The user can make a loose-pinch gesture, such as holding a small virtual ball between two fingers in space, and a tight-pinch gesture, such as holding an even smaller virtual ball between two fingers in space. You can make gestures. As shown in (a) of FIG. 4, when the user makes a loose-pinch gesture at one vertex, the electronic device 100 displays a cursor corresponding to the vertex and at least one control point adjacent to the cursor. You can. At this time, the cursor can impose restrictions on the position of the corresponding vertex. Here, the cursor may be in the shape of a sphere. Meanwhile, as shown in (b) of FIG. 4, when the user makes a tight-pinch gesture at one vertex, the electronic device 100 may enable modification of the position of the vertex.

Meanwhile, as shown in (c) of FIG. 3, at least some of the five fingers may be spaced apart from each other in the loose-grab gesture, and as shown in (d) of FIG. 3, in the tight-grab gesture. In a gesture, five fingers may be pressed against each other. Here, the user can make a loose-grab gesture between five fingers in space, such as holding a small cylinder within the grasp, and a tight-grab gesture in space, such as holding a much smaller cylinder within the grasp. You can take it. As shown in (c) of FIG. 4, when the user makes a loose-grab gesture at one vertex, the electronic device 100 displays a cursor corresponding to the vertex and at least one control point adjacent to the cursor. can do. At this time, the cursor can impose restrictions on the position and normal of the corresponding vertex. Here, the cursor may be in the shape of a cylinder. Meanwhile, as shown in (d) of FIG. 4, when the user makes a tight-grab gesture at one vertex, the electronic device 100 may enable modification of the position and normal of the vertex.

FIGS. 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, and 16 are diagrams of the electronic device 100 according to various embodiments. These are illustrations to explain two-hand interaction based on the gestures of Figures 3 and 4.

Referring to FIG. 5, the electronic device 100 can generate a straight line in space. Specifically, as shown in (a) of FIG. 5, the electronic device 100 can detect a tight-pinch gesture for each of the user's two hands. In this case, the electronic device 100 may generate vertices at the positions of both hands and create straight lines connecting the vertices. In some embodiments, the electronic device 100 generates a straight line only when the distance between the two hands is less than or equal to a predetermined distance, and when the distance between the hands exceeds the predetermined distance, the electronic device 100 generates a straight line. -You can also ignore the pinch gesture. Additionally, as shown in (b) of FIG. 5 , when the distance between both hands increases, the electronic device 100 may extend the corresponding straight line by moving vertices to the positions of both hands. At this time, as shown in (c) of FIG. 5, when the tight-pinch gesture is released, the electronic device 100 can maintain the corresponding straight line in space.

Referring to FIG. 6, the electronic device 100 can generate a curve in space. Specifically, as shown in (a) of FIG. 6, the electronic device 100 can detect a tight-grab gesture at each of the vertices on a straight line for both hands of the user. In this case, as shown in (b) of FIG. 6, as the directions of both hands are switched, the electronic device 100 may bend a straight line to create a curve. At this time, as shown in (c) of FIG. 6, when the tight-grab gesture is released, the electronic device 100 can maintain the corresponding curve in space.

Referring to FIG. 7, the electronic device 100 can remove lines from space. Specifically, as shown in (a) of FIG. 7 , the electronic device 100 may detect a tight-grab gesture at each vertex on a line for both hands of the user. In this case, as shown in (b) of FIG. 7, the gap between both hands is reduced to a predetermined distance or less, and when the corresponding tight-grab gesture is released as shown in (c) of FIG. 7, the electronic device (100) can remove the line from space. Figure 7 shows an example in which curves are removed, but similarly, straight lines will also be removed.

Referring to FIG. 8, the electronic device 100 can change lines in space. Specifically, as shown in (a) of FIG. 8, the electronic device 100 may detect a tight-pinch gesture at one vertex on a line for one hand of the user. In this case, as shown in (b) of FIG. 8, as the hand moves, the electronic device 100 may move the corresponding vertex. Here, the electronic device 100 can change the corresponding line while maintaining the normal line at the corresponding vertex. In other words, the line is connected to another vertex, so it can change as the vertex moves. At this time, as shown in (c) of FIG. 8, when the tight-pinch gesture is released, the electronic device 100 can maintain the corresponding line. Figure 8 shows an example in which a curve is changed, but similarly, a straight line will also be changed.

Referring to FIG. 9, the electronic device 100 can change lines in space. Specifically, as shown in (a) of FIG. 9 , the electronic device 100 may detect a tight-grab gesture at one vertex on a line for one hand of the user. In this case, as shown in (b) of FIG. 9, as the hand moves, the electronic device 100 may move the corresponding vertex. Here, the electronic device 100 can change the corresponding line along with the normal line at the corresponding vertex. In other words, the line is connected to another vertex, so it can change as the vertex moves. At this time, as shown in (c) of FIG. 9, when the tight-grab gesture is released, the electronic device 100 can maintain the corresponding line. Figure 9 shows an example in which a curve is changed, but similarly, a straight line will also be changed.

Referring to FIG. 10, the electronic device 100 can connect lines in space. Specifically, as shown in (a) of FIG. 10, the electronic device 100 may detect a tight-pinch gesture at one vertex on a line for one hand of the user. Through this, the electronic device 100 can move the corresponding vertex as the hand moves. Here, the electronic device 100 can change the corresponding line while maintaining the normal line at the corresponding vertex. And, as shown in (b) of FIG. 10, when the corresponding vertex is moved to another vertex on another line, the electronic device 100 may merge the corresponding vertex with the other vertex and connect the corresponding line to the other line. At this time, as shown in (c) of FIG. 10, when the tight-pinch gesture is released, the electronic device 100 can maintain the corresponding line. Figure 10 shows an example in which curves are connected, but similarly, curves and straight lines may be connected, or straight lines may be connected.

Referring to FIG. 11, the electronic device 100 can divide one line into two lines in space. Specifically, as shown in (a) of FIG. 11, the electronic device 100 can detect a tight-pinch gesture within a line for one hand of the user. In this case, as shown in (b) of FIG. 11, the electronic device 100 may generate a vertex at the position of the hand on the line. As a result, the line can be divided into two lines with the vertex in between. At this time, as shown in (c) of FIG. 11, when the tight-pinch gesture is released, the electronic device 100 can maintain the corresponding lines. Figure 11 shows an example in which one curve is divided into two curves, but similarly, one straight line will be divided into two straight lines.

Referring to FIG. 12, the electronic device 100 can rearrange lines in space. Specifically, as shown in (a) of FIG. 12, the electronic device 100 can detect a tight-grab gesture within a line for one hand of the user. In this case, as shown in (b) of FIG. 12, the electronic device 100 may move the line as the hand moves, or rotate the line as the hand rotates. Here, the line can be moved or rotated without deformation. If the line in question forms a closed figure with at least one other line, the electronic device 100 can move or rotate the entire figure without deformation as the hand moves. At this time, as shown in (c) of FIG. 12, when the tight-grab gesture is released, the electronic device 100 can maintain the corresponding line.

Referring to FIG. 13, the electronic device 100 can separate a line from a vertex in space. Specifically, as shown in (a) of FIG. 13, the electronic device 100 detects a loose-pinch gesture at one vertex for one hand of the user (e.g., the non-dominant hand (NDH)), For the user's other hand (e.g., dominant hand (DH)), a tight-grab gesture can be detected on a line connected to that vertex. In this case, as shown in (b) of FIG. 13, as the other hand moves, the electronic device 100 may separate the corresponding line from the corresponding vertex. Here, as the other hand moves, the line will also move, and the electronic device 100 may create a new vertex at the exposed end of the line. At this time, as shown in (c) of FIG. 13, when the loose-pinch gesture and the tight-grab gesture are released, the electronic device 100 can maintain the corresponding line.

Referring to FIG. 14, the electronic device 100 can separate a vertex from a line in space. Specifically, as shown in (a) of FIG. 14, the electronic device 100 detects a loose-grab gesture in one line for one hand of the user (e.g., non-dominant hand (NDH)), For the user's other hand (e.g. the dominant hand (DH)), tight-pinch gestures can be detected at the vertices connected to that line. In this case, as shown in (b) of FIG. 14, as the other hand moves, the electronic device 100 may separate the corresponding vertex from the corresponding line. Here, as the other hand moves, the corresponding vertex will also move, and the electronic device 100 may create a new vertex at the exposed end of the line. And, if at least one other line is connected to that vertex, the other line will be moved along with that vertex. At this time, as shown in (c) of FIG. 14, when the loose-grab gesture and the tight-pinch gesture are released, the electronic device 100 can maintain the corresponding vertex.

Referring to FIG. 15, the electronic device 100 can change the line connected to the vertex based on the vertex in space. Specifically, as shown in (a) of FIG. 15 , the electronic device 100 may detect a loose-pinch gesture at the vertex for one hand (eg, non-dominant hand (NDH)) of the user. In this case, the electronic device 100 may display a cursor indicating the position of the vertex and at least one control point adjacent to the cursor. Then, as shown in (b) of FIG. 15, the electronic device 100 may detect a tight-pinch gesture at the control point for the user's other hand (eg, dominant hand (DH)). In this case, as shown in (c) of FIG. 15, the electronic device 100 may change the line connected to the vertex as the other hand moves. Here, the electronic device 100 may maintain the position of the vertex and change the normal line at the vertex to correspond to the virtual line connecting the vertex and the control point. At this time, although not shown, when the loose-pinch gesture and the tight-pinch gesture are released, the electronic device 100 can maintain the corresponding vertex and the corresponding line.

Referring to FIG. 16, the electronic device 100 can change the line connected to the vertex based on the vertex in space. Specifically, as shown in (a) of FIG. 16, the electronic device 100 may detect a loose-grab gesture at the vertex for one hand (eg, non-dominant hand (NDH)) of the user. In this case, the electronic device 100 may display a cursor indicating the position and normal of the vertex and at least one control point adjacent to the cursor. Then, as shown in (b) of FIG. 16, the electronic device 100 may detect a tight-pinch gesture at the control point for the user's other hand (eg, dominant hand (DH)). In this case, as shown in (c) of FIG. 16, the electronic device 100 may change the line connected to the vertex as the other hand moves. Here, the electronic device 100 can maintain the normal line at the vertex. At this time, although not shown, when the loose-grab gesture and the tight-pinch gesture are released, the electronic device 100 can maintain the corresponding vertex and the corresponding line.

In this way, the electronic device 100 can create a virtual reality 3D curved network using the user's two-handed interaction. Specifically, while the user is wearing the electronic device 100, the electronic device 100 may create a virtual 3D curved network based on at least one hand gesture of the user on the computer. Gestures are based on pinch gestures and grab gestures and may include loose-pinch gestures, tight-pinch gestures, loose-grab gestures, and tight-grab gestures. That is, the electronic device 100 can create a 3D curved network in space by creating and modifying lines according to a gesture of at least one hand of the user.

FIG. 17 is a flowchart illustrating a method of operating the electronic device 100 according to various embodiments.

Referring to FIG. 17, the electronic device 100 may detect at least one hand gesture of the user in space in step 210. Specifically, while the user is wearing the electronic device 100, the processor 170 may detect at least one hand gesture of the user on the computer. At this time, when the electronic device 100 is implemented as a near-eye display device, the processor 170 captures an image in front of the user through the camera module 110 and monitors at least one hand of the user within the image. You can. Here, when at least one hand of the user makes a predetermined gesture, the processor 170 may detect it. As described with reference to Figures 3 and 4, gestures are based on pinch gestures and grab gestures and may include loose-pinch gestures, tight-pinch gestures, loose-grab gestures, and tight-grab gestures. .

Next, in step 220, the electronic device 100 may create a virtual 3D curved network according to at least one hand gesture of the user in space. Specifically, as described with reference to FIGS. 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, and 16, the processor 170 ) can create a new line or modify a created line according to a gesture of at least one hand of the user. Here, each line can be defined to connect two vertices, and each line can be a straight line or a curve. If the operation of the electronic device 100 to create the 3D curved network does not end in step 230, the electronic device 100 may repeatedly perform

steps

210 and 220 to continuously create the 3D curved network. Through this, the 3D curved network can be composed of multiple lines.

According to the present disclosure, a user can intuitively create a 3D curved network using two-handed interaction in virtual reality through the electronic device 100. Specifically, the electronic device 100 allows the user to easily create and modify lines by directly viewing them in real size in virtual reality through gestures with both hands. Therefore, 3D curved networks can be created with high expressiveness and precision in virtual reality.

In short, the present disclosure provides a method and device for producing a virtual reality 3D curved network using two-handed interaction.

According to the present disclosure, a method of creating a virtual reality 3D curved network using two-handed interaction of an electronic device 100 includes detecting a gesture of at least one hand of a user in space (step 210), and generating a plurality of gestures according to the gesture in space. It may include a step (step 220) of creating a 3D curved network composed of lines, where each line is defined between two vertices.

In various embodiments, the gesture may be based on a pinch gesture with two fingers and a grab gesture with five fingers.

In various embodiments, the gesture may include a loose-pinch gesture with two fingers spaced apart from each other, a tight-pinch gesture with two fingers pressed against each other, and at least some of the five fingers spaced apart from each other. It may include a loose-grab gesture and a tight-grab gesture in which five fingers are pressed against each other.

In various embodiments, the step of creating a 3D curve network (step 220) is to generate vertices at the positions of both hands when a tight-pinch gesture is detected for each of the user's two hands and create a straight line connecting the vertices. It may include the step of (see FIG. 5).

In various embodiments, creating a 3D curved network (step 220) may include extending a straight line while moving vertices to the positions of the hands as the spacing between the hands increases (see FIG. 5 ).

In various embodiments, the step of creating a 3D curve network (step 220) is to create a curve by bending the straight line as the directions of both hands are switched when a tight-grab gesture is detected at each of the vertices on the straight line for both hands of the user. It may include a generating step (see FIG. 6).

In various embodiments, the step of creating a 3D curved network (step 220) is that when a tight-grab gesture is detected at each of the vertices on the line for both hands of the user, the gap between the two hands is reduced to a predetermined distance or less. A step of removing the line may be included (see FIG. 7).

In various embodiments, the step of creating a 3D curved network (step 220) includes, when a tight-pinch gesture is detected at one vertex on a line for one hand of the user, moving the vertex as one hand moves to create a vertex. It may include a step of changing the line while maintaining the normal line (see FIG. 8).

In various embodiments, the step of creating a 3D curve network (step 220) includes, when a tight-grab gesture is detected at one vertex on a line for one hand of the user, moving the vertex as one hand moves to create a vertex. It may include a step of changing the line along with the normal line (see FIG. 9).

In various embodiments, creating a 3D curved network (step 220) may include connecting a line to another line while merging the vertex to another vertex if the vertex is moved to another vertex on another line. (See Figure 10).

In various embodiments, the step of creating a 3D curve network (step 220) includes, when a tight-pinch gesture is detected within a line for one hand of the user, creating a vertex at the position of one hand on the line, thereby creating a line. It may include dividing into two straight lines or two curves (see FIG. 11).

In various embodiments, creating the 3D curved network (step 220) may include moving the line as one hand moves when a tight-grab gesture is detected within the line for one hand of the user. (see Figure 12).

In various embodiments, creating the 3D curved network (step 220) may include rotating the line as one hand rotates when a tight-grab gesture is detected within the line for one hand of the user. (see Figure 12).

In various embodiments, creating a 3D curved network (step 220) includes detecting a loose-pinch gesture at one vertex for one hand of the user, and a tight-pinch gesture at a line connecting the vertices for the other hand of the user. When a grab is detected, the step of separating the connected line from the vertex while moving the connected line as the other hand moves (see FIG. 13).

In various embodiments, creating a 3D curved network (step 220) includes detecting a loose-grab gesture in one line for one hand of the user and a tight-grab gesture at a vertex connected to the line for the other hand of the user. When a pinch gesture is detected, the step of separating the connected vertices from the line while moving the connected vertices as the other hand moves (see FIG. 14).

In various embodiments, creating a 3D curved network (step 220) includes, when a loose-pinch gesture is detected at a vertex for one hand of the user, a cursor indicating the position of the vertex and at least one control point adjacent to the cursor. displaying, and when a tight-pinch gesture is detected at the control point for the user's other hand, the normal at the vertex changes correspondingly to the virtual line connecting the vertex and the control point as the other hand moves. It may include the step of changing the connected line (see FIG. 15).

In various embodiments, creating a 3D curved network (step 220) includes, when a loose-grab gesture is detected at a vertex for one hand of the user, a cursor indicating the position and normal of the vertex and at least one adjacent to the cursor. displaying a control point, and when a tight-pinch gesture is detected at the control point for the user's other hand, changing the line connected to the vertex while maintaining the normal at the vertex as the other hand moves. There is (see Figure 16).

According to the present disclosure, the electronic device 100 for creating a virtual reality 3D curved network using two-handed interaction, uses the camera module 110, the display module 140, and the camera module 110 to capture at least the user's image in space. A processor configured to detect a gesture of a hand and, through the display module 140, create a 3D curved network consisting of a plurality of lines according to the gesture in space, where each line is defined between two vertices. It can be included.

The system described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, the systems and components described in the embodiments include a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), and a programmable logic unit (PLU). It may be implemented using one or more general-purpose or special-purpose computers, such as a logic unit, microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. The software and/or data may be embodied in any type of machine, component, physical device, computer storage medium or device for the purpose of being interpreted by or providing instructions or data to the processing device. there is. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

Methods according to various embodiments may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. At this time, the medium may continuously store a computer-executable program, or temporarily store it for execution or download. In addition, the medium may be a variety of recording or storage means in the form of a single or several pieces of hardware combined. It is not limited to a medium directly connected to a computer system and may be distributed over a network. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, And there may be something configured to store program instructions, including ROM, RAM, flash memory, etc. Additionally, examples of other media include recording or storage media managed by app stores that distribute applications, sites or servers that supply or distribute various other software, etc.

The various embodiments of this document and the terms used herein are not intended to limit the technology described in this document to a specific embodiment, and should be understood to include various changes, equivalents, and/or replacements of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar components. Singular expressions may include plural expressions, unless the context clearly indicates otherwise. In this document, expressions such as “A or B”, “at least one of A and/or B”, “A, B or C” or “at least one of A, B and/or C” refer to all of the items listed together. Possible combinations may be included. Expressions such as "first", "second", "first" or "second" can modify the corresponding components regardless of order or importance, and are only used to distinguish one component from another. It does not limit the components. When a component (e.g. a first) component is said to be "connected (functionally or communicatively)" or "connected" to another (e.g. a second) component, it means that the component is connected to the other component. It may be connected directly to a component or may be connected through another component (e.g., a third component).

The term “module” used in this document includes a unit comprised of hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be an integrated part, a minimum unit that performs one or more functions, or a part thereof. For example, a module may be comprised of an application-specific integrated circuit (ASIC).

According to various embodiments, each component (eg, module or program) of the described components may include a single entity or a plurality of entities. According to various embodiments, one or more of the components or steps described above may be omitted, or one or more other components or steps may be added. Alternatively or additionally, multiple components (eg, modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component of the plurality of components prior to integration. According to various embodiments, steps performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the steps may be executed in a different order, omitted, or , or one or more other steps may be added.

Claims

In a method of creating a virtual reality 3D curved network using two-handed interaction of an electronic device,

detecting a gesture of at least one hand of a user in space; and

Creating a 3D curved network consisting of a plurality of lines according to the gesture in the space, where each line is defined between two vertices.

Including,

The gesture is based on a pinch gesture with two fingers and a grab gesture with five fingers,

method.
According to claim 1,

The gesture is,

A loose-pinch gesture in which the two fingers are spaced apart from each other,

A tight-pinch gesture in which the two fingers are pressed against each other,

A loose-grab gesture in which at least some of the five fingers are spaced apart from each other, and

Tight-grab gesture with the five fingers pressed against each other

Including,

method.
According to claim 2,

The step of producing the 3D curved network is,

When the tight-pinch gesture is detected for each of the user's two hands, generating vertices at the positions of the two hands and creating a straight line connecting the vertices; or

When the gap between the hands increases, extending the straight line while moving the vertices to the positions of the hands.

Containing at least one of

method.
According to claim 2,

The step of producing the 3D curved network is,

When the tight-grab gesture is detected at each of the vertices on a straight line for both hands of the user, bending the straight line to create a curve as the directions of both hands are switched.

Including,

method.
According to claim 2,

The step of producing the 3D curved network is,

When the tight-grab gesture is detected at each of the vertices on the line for both hands of the user, removing the line as the gap between the hands is reduced to a predetermined distance or less.

Including,

method.
According to claim 2,

The step of producing the 3D curved network is,

When the tight-pinch gesture is detected at one vertex on a line for one hand of the user, changing the line while maintaining the normal at the vertex by moving the vertex as the one hand moves; or

When the vertex is moved to another vertex on another line, merging the vertex with the other vertex and connecting the line to the other line.

Containing at least one of

method.
According to claim 2,

The step of producing the 3D curved network is,

When the tight-grab gesture is detected at a vertex on a line with respect to one hand of the user, moving the vertex as the one hand moves to change the line along with the normal at the vertex;

When the tight-grab gesture is detected within a line for one hand of the user, moving the line as the one hand moves; or

When the tight-grab gesture is detected within a line for one hand of the user, rotating the line as the one hand rotates.

Containing at least one of

method.
According to claim 2,

The step of producing the 3D curved network is,

When the tight-pinch gesture is detected within a line for one hand of the user, dividing the line into two straight lines or two curves by creating a vertex at the position of the one hand on the line.

Including,

method.
According to claim 2,

When the loose-pinch gesture is detected at one vertex for one hand of the user, and the tight-grab is detected at a line connected to the vertex for the other hand of the user, as the other hand moves, the connected separating the connected line from the vertex while moving the line; or

If the loose-grab gesture is detected in one line for one hand of the user, and the tight-pinch gesture is detected at a vertex connected to the line for the other hand of the user, as the other hand moves, the separating the connected vertices from the line while moving the connected vertices

Containing at least one of

method.
According to claim 2,

The step of producing the 3D curved network is,

When the loose-pinch gesture is detected at a vertex for one hand of the user, displaying a cursor indicating the position of the vertex and at least one control point adjacent to the cursor; and

When the tight-pinch gesture is detected at the control point for the user's other hand, the normal at the vertex changes as the other hand moves corresponding to the virtual line connecting the vertex and the control point. Steps to change the line connected to the vertex

Including,

method.
According to claim 2,

When the loose-grab gesture is detected at a vertex for one hand of the user, displaying a cursor indicating the position and normal of the vertex and at least one control point adjacent to the cursor; and

When the tight-pinch gesture is detected at the control point for the user's other hand, changing a line connected to the vertex while maintaining the normal at the vertex as the other hand moves.

Including,

method.
In an electronic device for creating a virtual reality 3D curved network using two-handed interaction,

camera module;

display module; and

Through the camera module, a gesture of at least one hand of the user is detected in space, and through the display module, a plurality of lines are formed according to the gesture in the space, where each line is defined between two vertices. A processor configured to produce a 3D curved network

Including,

The gesture is based on a pinch gesture with two fingers and a grab gesture with five fingers,

Electronic devices.
A computer program stored in a non-transitory computer-readable recording medium for executing a method of creating a virtual reality 3D curved network using two-handed interaction on an electronic device, comprising:

The above method is,

detecting a gesture of at least one hand of a user in space; and

Creating a 3D curved network consisting of a plurality of lines according to the gesture in the space, where each line is defined between two vertices.

Including,

The gesture is based on a pinch gesture with two fingers and a grab gesture with five fingers,

computer program.