WO2021133081A1

WO2021133081A1 - Method, apparatus, and program for providing user interface for arrangement of virtual object

Info

Publication number: WO2021133081A1
Application number: PCT/KR2020/019059
Authority: WO
Inventors: 손범준; 권영호
Original assignee: 손범준; 권영호
Priority date: 2019-12-27
Filing date: 2020-12-24
Publication date: 2021-07-01
Also published as: KR20210084073A; KR102351900B1

Abstract

Provided are a method, apparatus, and program for providing, by a client device, a user interface for arrangement of a virtual object The method for providing a user interface comprises: a step (S2100) for providing a screen of a user space in a direction directed by the client device; a step (S2200) for outputting a selected object to be fixed and located at one region of the screen, according to a selection request of a user for a specific object; a step (S2300) for, when the client device is moved by the user while the selected object is selected, providing the screen of the user space according to the direction directed by the client device, wherein the selected object is output to be fixed and located at the one region of the screen; and a step (S2400) for arranging the selected object in the user space according to an arrangement request of the user for the selected object.

Description

Method, apparatus and program for providing a user interface for virtual object placement

The present invention relates to a method of providing a user interface, and more particularly, to a method of providing a user interface for disposing a virtual object in a virtual space.

Efforts to create a virtual space by arranging virtual objects due to the recent increase in various services to which extended reality (XR) is applied, including virtual reality (VR) and augmented reality (AR, Augmented Reality) are continuing

The conventional method of arranging an object in a virtual space is different from a method in which a user arranges an object in reality, such as inputting spatial coordinates with respect to an arrangement position.

In addition, due to the above-described complicated object placement method, the conventional user interface for placement of a virtual object in a virtual space has a problem in that the operation method is complicated and a separate device such as a keyboard is required.

An object of the present invention is to provide a user interface through which a user can easily arrange a virtual object based on a spherical coordinate system with a simple manipulation in a virtual space, similar to a method in which a user arranges a real object in a real space.

Another object of the present invention is to provide a client device capable of providing the user interface without requiring a separate device for a user to place a virtual object in a virtual space.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

A method for providing a user interface according to an aspect of the present invention for solving the above-described problems includes providing a user space screen in a direction directed by a client device; outputting the selection object to be fixedly positioned on one area of the screen in response to a user's request for selection of a specific object; When the client device is moved by the user while the selection object is selected, the user space screen is provided according to the direction the client device is oriented, and the selection object is fixedly positioned on the one area of the screen. outputting; and arranging the selection object in the user space in response to a user's request for placement of the selection object, wherein the user space includes one or more virtual objects arranged by the user.

Also, in another embodiment, the location of the selection object on a sphere at a specific distance from the origin is changed according to the movement of the client device, and the origin is the arrangement position of the client device in a state in which the selection object is selected. or a user location calculated from the placement location of the client device.

Also, in another embodiment, the selection request includes the user touching a specific object on the screen or the user touching a second icon in a state where the first icon is located on the specific object, and the first icon is fixed to one area of the screen, and the second icon is located in another area of the screen spaced apart from the first icon.

Also, in another exemplary embodiment, the placement request includes the user touching the screen while the selection object is selected.

Also, in another embodiment, according to the user's request for changing the distance, changing the distance between the origin and the selection object; further includes.

Also, in another embodiment, the request for changing the distance includes a user tapping or dragging a third icon located in an area of the screen while the selection object is selected.

Also, in another embodiment, the step of outputting the selection object to the screen includes outputting the selection object to be fixedly positioned on an area of the screen, connecting the selection object and the third icon, and connecting the An object indicating a distance between the selection objects is further output.

Also, in another embodiment, according to a user's request for size change, changing the size of the selected object; further includes.

Further, in another embodiment, according to the user's rotation request, the step of rotating the selected object; further includes.

In addition, an apparatus for providing a user interface according to an aspect of the present invention includes: an input unit for receiving a user's selection request for a specific object or a user's placement request for a selection object selected by the user; a detection unit for detecting the position, direction, or operation information of the client device; a control unit that moves the selection object in the user space according to the movement of the client device or a user's manipulation, and arranges the selection object in the user space according to the arrangement request; and an output unit for outputting, by the control unit, a user space screen in a direction the client device is oriented, and outputting the selection object to be fixedly positioned in an area of the screen according to the inputted user's selection request; and, the user space includes one or more objects arranged by a user.

A user interface providing program according to another embodiment of the present invention is combined with hardware to execute the above-mentioned user interface providing method, and is stored in a medium.

Other specific details of the invention are included in the detailed description and drawings.

According to the present invention as described above, it has various effects as follows.

First, similar to object placement in real space, a user can select a virtual object with simple manipulations in virtual space, and move the selected selected object while maintaining a specific distance from the client device only by moving the client device without any other manipulations. and the distance from the client device can be changed with a simple operation, so that a specific object can be easily disposed at a desired location.

Second, since the size or direction of the selected selection object can be changed with a simple manipulation, a user space can be easily formed by arranging one or more virtual objects in the virtual space without a separate device or complicated calculation.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 to 4 are diagrams for explaining an embodiment of a method of forming a virtual space by arranging a virtual object through a user interface according to an embodiment of the present invention.

5 is a flowchart of a method for providing a user interface according to an embodiment of the present invention.

6 is a block diagram of a client device providing a user interface according to an embodiment of the present invention.

7A and 7B are diagrams for explaining provision of a user space screen in a direction directed by a client device according to an embodiment of the present invention.

8A and 8B are diagrams for explaining a selection request by a user touching an object on a screen according to an embodiment of the present invention.

9A and 9B are diagrams for explaining a selection request through a first icon and a second icon according to an embodiment of the present invention.

10A and 10B are diagrams for explaining a screen according to movement of a client device in a state in which a selection object is selected according to an embodiment of the present invention.

11A and 11B are diagrams for explaining an arrangement request by a user's touch on the screen according to an embodiment of the present invention.

12 is a flowchart of a method for providing a user interface further including a distance change step according to a user's distance change request according to an embodiment of the present invention.

13A and 13B are diagrams for explaining a distance change request and a distance display object through a third icon according to an embodiment of the present invention.

14 is a flowchart of a method for providing a user interface further including a resizing step in response to a user's size change request according to an embodiment of the present invention.

15A and 15B are diagrams for explaining a size change request by a user pinch-in/out according to an embodiment of the present invention.

16 is a flowchart of a method for providing a user interface further comprising a rotation step according to a user's rotation request according to an embodiment of the present invention.

17A and 17B are diagrams for explaining a roll rotation of a selection target according to an embodiment of the present invention.

18A and 18B are diagrams for explaining yaw rotation of a selection target according to an embodiment of the present invention.

19A and 19B are diagrams for explaining pitch rotation of a selection target according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully understand the scope of the present invention to those skilled in the art, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. As used herein, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe the correlation between a component and other components. A spatially relative term should be understood as a term that includes different directions of components during use or operation in addition to the directions shown in the drawings. For example, when a component shown in the drawing is turned over, a component described as “beneath” or “beneath” of another component may be placed “above” of the other component. can Accordingly, the exemplary term “below” may include both directions below and above. Components may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation.

In the present specification, the term 'object' refers to a virtual object disposed in a virtual space including virtual reality (VR) or augmented reality (AR). The object includes a computer graphic (CG) image of a specific object or an image of a real object or person, but is not limited thereto, and includes all objects that can be placed in a virtual space.

In this specification, 'user space' refers to a virtual space including one or more virtual objects arranged by a user. The virtual space includes a VR virtual space in which an object or a background of the real world is provided only as a CG image, or an AR virtual space that is an image of a virtual object provided on an image of the real space.

In the present specification, a 'selection object' refers to an object selected by a user's selection request from among one or more objects existing in the user space. For example, the selection object may be moved in the user space according to the user's movement of the client device or the user's manipulation until it is placed at a location on the virtual space desired by the user after selection.

In this specification, a 'client device' refers to any device including a communication function that users can use by installing a program (or application). That is, the client device may include one or more of a telecommunication device such as a smartphone, tablet, PDA, laptop, smart watch, smart camera, etc., a VR/AR device such as a remote controller, VR/AR glasses, VR/AR headset, etc. However, the present invention is not limited thereto.

As used herein, the term 'computer' includes various devices capable of performing arithmetic processing. For example, computers include desktop PCs, notebooks (Note Books) as well as smart phones, tablet PCs, cellular phones, PCS phones (Personal Communication Service phones), synchronous/asynchronous A mobile terminal of International Mobile Telecommunication-2000 (IMT-2000), a Palm Personal Computer (PC), a Personal Digital Assistant (PDA), and the like may also be applicable.

Hereinafter, before describing an embodiment of the present invention with reference to the accompanying drawings, an embodiment of a method of forming a virtual space by arranging a virtual object through a user interface according to an embodiment of the present invention will be described. .

1 to 4 are diagrams for explaining an embodiment of a method of forming a virtual space by arranging a virtual object through a user interface according to an embodiment of the present invention. 1 to 4 , according to an embodiment of a method of forming a virtual space by arranging a virtual object through a user interface according to an embodiment of the present invention, the user client device 300 provides a user space ( 20) generating or executing (S210); performing a position change of the selection object 450 selected by the user 200 in response to the movement of the user client device or the user's manipulation (S310); and arranging the selection object in the user space according to the user's request for placement of the selection object (S410). Hereinafter, a detailed description of each step will be described.

The user client device 300 creates or executes the user space 20 (S210). That is, according to an embodiment, the user client device creates a user space that is a virtual space in which the user will place a virtual object or reconnects to the previously created user space in response to a user's request.

In one embodiment, the user space 20 is a virtual space including one or more virtual objects arranged by the user, and is an AR virtual space generated to receive and correspond to image data of a real space as well as a VR virtual space. includes

The user client device changes the location of the selection object 450 selected by the user in response to the movement of the user client device or the manipulation of the user (S310). The selection object 450 is a specific object in the user space selected by the user's selection request, and the selection request includes all inputs of the user's specific object that can be input to the user client device, which will be described in detail. It will be described later.

In one embodiment, the movement of the client device may include not only a movement of changing the position (coordinate) of the client device, but also a change of direction, ie, a rotational movement of the client device.

In an embodiment, the change of the selection object is a state in which the location of the selection object in the user space corresponding to the movement of the client device or the manipulation of the user, the distance of the selection object from the origin is changed, or the position in the user space is fixed includes changing the shape, size, or orientation of the selected object. Hereinafter, it will be described in detail.

In an embodiment, the change in the location of the selection object in the user space is changing the arrangement location of the selection object in the user space, which is a three-dimensional virtual space, wherein the selection object moves or moves the user client device while the selection object is selected. The location is changed according to the user's manipulation. In addition, the arrangement position includes not only position (coordinate) information, but also arrangement information regarding the direction of the selection target.

In one embodiment, referring to FIG. 2 , the selection object 450 responds to the movement of the user client device while maintaining a specific distance with respect to a specific direction of the user client device 300 in the user space 20 . can be moved

For example, when the selection object is located at a distance of 1 m perpendicular to the center of the rear surface (opposite to the user) of the user client device, the selection object moves (rotation, position change, etc.) of the user client device by the user 200 . ), the position in the user space may be changed to correspond to this. That is, in this case, there is an effect that the user can change the position of the selection object only by moving (including rotation) the client device without a separate manipulation of the arrangement of the selection object. Also, as will be described later, the distance between the selection target and the user client device may be changed according to a user's manipulation.

Also, in one embodiment, referring to FIG. 2 , the direction of the selection target may be fixed with respect to the client device. For example, in the case where the front of the selection object before moving faces the client device and the position of the selection target is moved according to the movement of the client device, the front of the selection target after moving faces the client device after moving can be automatically rotated to Also, in another embodiment, the direction of the selection object may be fixed in the user space and only the location change may be performed according to the movement of the client device. In addition, as will be described later, the selection target is rotatable according to a user's manipulation, which will be described later in detail.

In addition, in an embodiment, the location of the selection object 450 may be changed in the user space to move on a spherical surface at a specific distance from the origin 150 according to the movement of the user client device or the user's manipulation in the selected state. have.

In an embodiment, the origin 150 includes the location of the user client device 300 in the state in which the selection object 450 is selected or the location of the user 200 calculated from the location of the user client device. .

FIG. 3 is a diagram for explaining a case where the origin 150 is the position of the user 200 calculated from the arrangement position of the user client device in a state in which the selection object is selected.

Referring to FIG. 3 , according to an embodiment, the origin, which is the user's location in a state in which the selection object is selected, may be calculated in consideration of the user's arm length or the user's client direction from the location of the user client device.

Also, in an embodiment, since the origin is a user's position in a state in which the selection object is selected, when the user moves in a state in which the selection object is selected, the user's position after the movement may be the origin.

Also, in one embodiment, the origin may include information about the direction as well as the coordinates of the user or the user's client device. That is, in FIG. 3 , the origin 150 may include the coordinates of the user 200 , the direction of the user client device 300 , or the direction of the user 200 .

Also, in one embodiment, referring to FIG. 3 , the selection target may be moved in the user space while maintaining a specific direction or a specific distance with respect to the user client device. For example, when a user extends an arm and holds a client device (eg, a smartphone) in his hand, and the selection object is located at a distance of 1 m perpendicular to the center of the rear surface of the client device, the user When the client device is moved with the arm extended to place the . , the selected object may be moved on a sphere at a distance of '1 m + the first user's arm length' from the origin, which is the user's location. That is, in this case, the selection object may move on a specific sphere with a distance (r value) of '1m + first user's arm length' from the origin. In addition, it goes without saying that the r value may be changed by the user's arm movement or user manipulation while the selection object is selected.

FIG. 4 is a diagram for explaining a case where the origin 150 is the arrangement position of the user client device 300 in a state in which the selection object is selected. Referring to FIG. 4 , according to an embodiment, the origin, which is the location of the user client device in a state in which the selection object is selected, may include information about the direction as well as the coordinates of the first user client device.

Also, since the origin is the arrangement position of the user client device when the selection object is selected, when the user moves the user client device while the selection object is selected, the position (including the direction) of the user client device after movement could be the starting point.

Also, in an embodiment, the location of the selection object 450 may be changed in the user space to move on a spherical surface at a specific distance from the origin according to the movement of the user client device or the user's manipulation in the selected state. That is, as shown in FIG. 4 , not only when the location of the client device is changed, but also when the location is fixed and only the direction is changed, the location of the selection object can be changed in the user space in response to the direction of the client device. have.

For example, when the user extends his arm and holds a client device (eg, a smart phone) in his hand, and the selected object is located at a distance of 1 m perpendicular to the center of the rear surface of the client device, the user If only the direction of the client device is rotated while the arm is extended to place the selection object at a desired location, the selection object may be moved on a spherical surface at a distance of '1 m' from the origin, which is the location of the client device.

On the other hand, when the user changes the location of the client device while extending his or her arm to place the selection object at a desired location, it is located at the center of the rear side of the client device on a spherical coordinate system with the location of the moved client device as the origin. The selection object may be located at a distance of 1 m orthogonal to the object. Also, it goes without saying that the r value may be changed by a user manipulation while the selection object is selected.

Also, in one embodiment, when the location and orientation of the client device and the user are the same (eg, VR headset), such as when the user wears the client device on the head without holding the hand, the origin is the client device location ( direction), of course.

In one embodiment, although not shown in the drawings, the method may further include receiving and applying change information on the distance between the origin 150 and the selection object 450 from the user 200 . This is to allow the selection object to move on the spherical surface of the changed distance by changing the distance in the selection object moved on the spherical surface at a specific distance from the origin as described above.

For example, in FIG. 4 , if the selection object is located at a distance of 1 m perpendicular to the center of the rear surface (opposite to the user) of the user client device, the distance of the selection object from the client device is set to 2 m according to the user's request. , and the position of the selected object may be changed in the user space to correspond to the movement including rotation and movement of the user client device by the user while maintaining it.

That is, by allowing the user to change the r value, which is the distance of the selection object from the origin (eg, the client device), there is an effect that can be freely and conveniently arranged for all positions in the user space. In addition, the distance change request includes all inputs for changing the distance of the user's selection target that can be input to the user's client device, which will be described in detail later.

In one embodiment, although not shown in the drawings, the method may further include receiving and applying change information on the selection object 450 from the user 200 . Also, in an embodiment, the change information of the selection object includes change information regarding the shape, size, or direction of the selection object in a state where the location of the selection object in the user space is fixed. That is, the user can freely change the shape, size, direction, etc. of the selected object by the user's manipulation while the specific object is selected, and it is also possible to change the shape by uploading a new object, which will be described later in detail.

The selection object is placed in the user space 20 in response to the user 200's request for placement of the selection object 450 (S410). That is, in the state in which the selection object is selected, when the selection object arrives at a location to be placed by the user (including movement of the client device or the user's manipulation), the selection of the selection object is released according to the user's placement request, , the selection object is arranged in the user space according to the position and direction of the selection object when the placement request is made.

That is, according to an embodiment, the disposed object is no longer moved in the user space by the movement of the client device or by the user manipulation, but may be moved when selected again by the user. In addition, the arrangement request includes all inputs for the user's selection target that can be input to the user's client device, which will be described in detail later.

As described above, an embodiment of a method of forming a virtual space by arranging a virtual object through a user interface according to an embodiment of the present invention has been described. However, the utilization of the method for providing a user interface according to an embodiment of the present invention is not limited to the embodiment of the method for forming a virtual space described above, and may be applied to all methods of forming a virtual space by arranging a virtual object. .

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. 7A and 7B, 8A and 8B, 9A and 9B, 10A and 10B, 11A and 11B, 13A and 13B, 15A and 15B, and FIGS. 17A and 17B, 18A and 18B, and 19A and 19B are diagrams for explaining a method of providing a user interface according to an embodiment of the present invention. to display the screen of the client device provided according to the user's manipulation in the left layer.

7A and 7B, FIGS. 8A and 8B, FIGS. 9A and 9B, FIGS. 10A and 10B, FIGS. 11A and 11B, FIGS. 13A and 13B, and FIGS. 15A and 15A and FIGS. 15B, 17A and 17B, 18A and 18B, and 19A and 19B according to an embodiment of the present invention, a user interface for placing a virtual rabbit object in the AR user space using a smartphone However, it is obvious that the present invention is not limited to the above embodiment, and can be applied to all the client devices, user spaces, and objects described above.

5 is a flowchart of a method for providing a user interface according to an embodiment of the present invention. Referring to FIG. 5 , a method of providing a user interface according to an embodiment of the present invention includes providing a screen of the user space 20 in a direction in which the client device 300 is oriented ( S2100 ); outputting the selection object 450 to be fixedly positioned in one area of the screen in response to a selection request of the user 200 for a specific object (S2200); When the client device is moved by the user while the selection object is selected, the user space screen is provided according to the direction the client device is oriented, and the selection object is fixedly positioned on the one area of the screen. outputting (S2300); and arranging the selection object in the user space according to the user's request for placement of the selection object (S2400). Hereinafter, a detailed description of each step will be described.

A screen of the user space 20 in the direction the client device 300 is oriented is provided (S2100). That is, the user space screen corresponding to the movement of the client device by the user is output to the output unit of the client device, and as described above, the user space includes a VR or AR user space.

7A and 7B are diagrams for explaining provision of a user space screen in a direction directed by a client device according to an embodiment of the present invention. Referring to FIGS. 7A and 7B , in one embodiment, a screen 382 for the user space 20 in the direction the client device is oriented is provided to the output unit of the smartphone 300 as the client device. In one embodiment, the user space is an AR virtual space, a virtual space in which one or more virtual objects 400, which are rabbits, are disposed in an image of a real space.

Specifically, when the smartphone, which is the client device 300, is moved from the position of FIG. 7A to the position of FIG. 7B by the user, the screen for the user space is output to the output unit of the smartphone to correspond thereto.

In response to a selection request of the user 200 for a specific object, the selection object 450 is output to be fixedly positioned in one area of the screen (S2200). In an embodiment, when a user space screen on which one or more objects are arranged is output on the client device, the object is selected in response to a user's request for selection of a specific object, and the selected selection object is fixed to an area of the screen of the client device and output to be positioned, and the selection request includes all inputs for selection of a specific object of the user that can be input to the user client device.

That is, according to an embodiment, the object in the user space is selected by the user, so that the position, size, or direction of the object in the user space may be changed in response to the movement of the client device or the manipulation of the user.

Referring to FIGS. 8A and 8B , in one embodiment, the red rabbit, which is the object 400 disposed in the user space 20 in FIG. 8A , selects the object 450 in FIG. 8B according to the user's selection request. It is output so that it is fixed in the center, which is one area of the smartphone screen 382 .

In an embodiment, when the user touches an area of the screen 382 of the client device, the method further includes outputting, by the client device, information indicating the touched location. For example, referring to FIG. 8A , when the user touches a red rabbit on the smartphone screen in the right layer, a white circle 384 for the position touched by the user is output on the smartphone screen, which is the left layer, and the touch position can inform the user.

Further, in an embodiment, 'touch' is a user's manipulation input to the client device, and includes a tap, a double tap, a drag, a slide, a press, and a touch and may be input to the client device including, but not limited to, Touch and Hold, Swipe, Rotate, Press&Drag, Pinch in/out, or Spread. Includes all user gestures.

In an embodiment, the selection request includes the user touching a specific object on the screen or the user touching the second icon while the first icon is located on the specific object.

8A and 8B are diagrams for explaining a selection request by a user touching an object on a screen according to an embodiment of the present invention. Referring to FIGS. a and b, in one embodiment, when a selection request is input in which the user touches 384 a red rabbit that is an object 400 output on the smartphone screen 382 in FIG. 8a, FIG. 8b The touched red rabbit becomes the selection object 450 and may be output to be fixed in the center of the screen.

9A and 9B are diagrams for explaining a selection request through a first icon and a second icon according to an embodiment of the present invention. In an embodiment, the first icon may be fixedly positioned on one region of the screen, and the second icon may be positioned on another region of the screen spaced apart from the first icon. For example, referring to FIGS. 9A and 9B , in an embodiment, the first icon 610 may be fixedly located in the center of an area of the smartphone screen 382 , and the second icon 620 . may be located at the bottom, which is another area of the screen.

In one embodiment, as shown in FIG. 9A , the user touches the second icon 620 while moving the smartphone 300 so that the first icon 610 is located on the red rabbit that is the object 400 to be selected. In this case, as shown in FIG. 9B , the red rabbit may be the selection target 450 .

In one embodiment, although not shown in the drawings, when the second icon is touched when the first icon is overlapped with the plurality of objects, the client device outputs an additional selection request for the plurality of objects to the user ; may be further included. In an embodiment, the additional selection request may include an additional selection request for one object among a plurality of objects or an additional selection request for all of the plurality of objects.

In an embodiment, the selection object 450 may further include a separate output to distinguish it from other unselected objects 400 disposed in the user space. For example, as shown in FIG. 8B , a highlight mark may be included in the selected object to distinguish it from the unselected object.

When the client device is moved by the user while the selection object is selected, the user space screen is provided according to the direction the client device is oriented, and the selection object is fixedly positioned on the one area of the screen. output (S2300). In an embodiment, even if the user moves the client device while a specific object is selected, the selection object is fixed to one area of the screen, and the direction in which the client device is moving is fixed to a screen other than the area where the selection object is output. Userspace can be printed.

That is, in one embodiment, as described above, the selection object may be moved in the user space according to the movement of the client device while maintaining a specific direction or a specific distance with respect to the client device, and the screen of the client device is displayed by the user. The movement of the selected object in space can be output.

10A and 10B are diagrams for explaining a screen according to movement of a client device in a state in which a selection object is selected according to an embodiment of the present invention. Referring to FIGS. 10A and 10B , in one embodiment, a red rabbit 450 that is a selection target selected by the user in the user space 20 is fixedly positioned in the center of an area of the smartphone screen 382 . is output, and when there is a movement of the smartphone 300 from FIG. 10A to FIG. 10B , the selection object is fixed at the center of the screen, but the screen of the user space 20 is output to correspond to the movement of the smartphone.

That is, in one embodiment, the location of the red rabbit 450 as the selection object is changed in the user space according to the movement of the smartphone, but the position of the white rabbit 400 as the unselected object is changed according to the movement of the smartphone. No change.

In an embodiment, when the user intends to move a plurality of objects in the user space at once, it is also possible to simultaneously select the plurality of objects.

In an embodiment, the selection of the plurality of objects includes a plurality of objects located on a path in which the first icon 610 is moved as the user moves the client device 300 while the user touches the second icon 620 . of may be selected.

For example, as the user moves the smartphone 300 in a state where the user touches and holds the second icon, object A, in which the first icon located in the center of the smartphone is disposed in the user space, When the user stops touching and holding the second icon after passing through B and C, the objects A, B, and C may be simultaneously selected.

In another embodiment, selecting the plurality of objects includes moving the first icon 610 to surround a predetermined area as the user moves the client device 300 while the user touches the second icon 620 . In this case, a plurality of objects located within the corresponding area may be selected.

For example, as the user moves the smartphone 300 in a state where the user touches and holds the second icon, the first icon located in the center of the smartphone is moved to draw a circle, and then the user When the touch and hold of the second icon is stopped, the objects A, B, and C arranged in the circle may be simultaneously selected.

In an embodiment, when a plurality of objects are selected at the same time, they are output to be fixedly positioned on an area of the screen of the user client device while maintaining the arrangement shape between the selected objects, and are positioned in the user space according to the movement of the client device may be changed. For example, all of A, B, and C selected at the same time may be grouped while maintaining the arrangement of each other, moved as a whole in the user space, and may be arranged according to a user's arrangement request.

In another embodiment, when a plurality of objects are simultaneously selected, all of the selected objects may be fixedly positioned on one area of the screen of the user client device. For example, each of A, B, and C selected at the same time may be overlapped and moved to be located in the center of the smartphone, and may be arranged according to a user's arrangement request.

In response to a user's request for placement of the selection object, the selection object is placed in the user space (S2400). That is, when the selection object arrives at a location that the user wants to place, the selection can be released in the user space at the location by releasing the selection according to the user's placement request, and the placement request can be input to the user's client device. Includes all inputs for the arrangement of the user's selection object. That is, according to an embodiment, after the selection object is disposed, the location is not changed according to the movement of the client device unless it is selected again.

11A and 11B , in one embodiment, when there is a user's placement request (eg, screen touch) for the red rabbit 450 that is the selection target in FIG. 11A , the selection target is placed on the selection target as shown in FIG. 11B . By canceling the selection and arranging in the user space 20 according to the location at the time of the request for placement, the red rabbit becomes the object 400 rather than the selection object.

In an embodiment, the placement request includes the user touching the screen while the selection object is selected. 11A and 11B are diagrams for explaining an arrangement request by a user's touch on the screen according to an embodiment of the present invention. 11A and 11B , in one embodiment, when the user touches 384 an area of the screen 382 while the red rabbit 450 is selected as shown in FIG. 11A , as shown in FIG. 11B , the selection object 450 may be disposed at a location at the time of a request for placement in the user space, and thus may become an object 400 rather than a selection object.

In an embodiment, the area touched for the placement request may be a screen other than the area where the selection object is located or the entire screen including the area where the selection object is located.

In one embodiment, although not shown in the drawings, the method may further include, in response to a user's request for duplication of the selected object, copying the selected object in user space. In one embodiment, the duplication of the selection object is to place an object having the same type, shape, size or direction as the selection object in the user space while maintaining the selection of the selection object, unlike the arrangement of the selection object described above, In this case, the user can freely move a specific selected object and conveniently arrange a plurality of duplicated objects through a copy request at each desired location.

For example, when the user selects the red rabbit 450 and moves the smartphone 300 and touches the fourth icon (request icon) for each location where the red rabbit is to be placed, the red rabbit 450 is displayed. may be placed in the center of the screen and the duplicated red rabbit object 400 may be disposed at each corresponding location in the user space.

In an embodiment, when a specific object is selected, placed, or copied according to the selection request, the placement request, or the copying request, the method may further include, by the user client device, outputting information that can notify the user of this. For example, image information including the aforementioned highlight display, voice information corresponding to selection/arrangement, or haptic (eg, vibration) information may be output.

As described above, a method of providing a user interface for selection according to a user operation, a change in a location of a selection object according to a movement of a client device, and an arrangement of a selection object according to a user operation have been described in detail.

Hereinafter, in a state in which the distance of the selection object is changed from the origin (eg, the client device) by a user manipulation, rather than movement of the client device in a state in which the selection object is selected, or the location of the selection object in the user space is fixed Changes in the shape, size or direction of the will be described later in detail.

12 is a flowchart of a method for providing a user interface further including a distance change step according to a user's distance change request according to an embodiment of the present invention. Referring to FIG. 12 , in one embodiment, the method further includes changing the distance between the origin and the selected object according to the user's request for changing the distance ( S2320 ). As described above, the origin includes an arrangement position of the user client device in a state in which the selection object is selected or a user position calculated from the arrangement position of the client device.

In an embodiment, the distance change request includes all inputs for changing the distance of the user's selection target that may be input to the user client device. The distance between the origin and the selection object in the user space may be changed according to the distance change request.

13A and 13B are diagrams for explaining a distance change request and a distance display object through a third icon according to an embodiment of the present invention. 13A and 13B , in an embodiment, the request for changing the distance includes a user tap or drag a third icon located in an area of the screen while the selection object is selected.

Specifically, when the user taps (384) the third icon 630 on the smartphone screen in FIG. 13A and drags it downward as shown in FIG. 13B (384), between the red rabbit as the selection target and the client (origin) distance can be narrowed. Similarly, the distance can be increased by dragging the third icon upward.

13A and 13B , according to an embodiment, the third icon 630 may be a change of the second icon 620 when a specific object is selected. That is, in an embodiment, when the object is not selected, the second icon for selecting the object is changed to a third icon for changing the distance of the selected object when the object is selected, and when the selection of the object is deselected and placed, it is again It may be changed to a second icon.

Also, in an embodiment, the output size of the selection object may be differently output on the screen of the client device according to the change in the distance. This is due to the perspective method according to the distance change, which is different from the change in the size of the selection object, which will be described later. That is, in one embodiment, the 'output size' of the selection object means the size of the selection object output image output on the screen of the client device, and the 'size' of the selection object is the size of the selection object itself existing in the user space 20 . It could mean size.

Also, in an embodiment, when the selection object is selected, an object connecting the third icon and the selection object and indicating the distance between the origin and the selection object may be further output. The distance display object may represent the distance as a numerical value 640 (eg, 208.92 cm in FIG. 13A and 85.17 cm in FIG. 13B ) or may be displayed using a figure.

13A and 13B , in an embodiment, the figure includes a dotted line 650 connecting a third icon and a selection object. For example, since the first icon 610 is located in the center of the screen, and the red rabbit 450, which is a selection target, is also fixedly located in the center of the screen, the dotted line 650 is the third icon 630 and the first The icon 610 may be connected.

Also, in an embodiment, the dotted line 650 may indicate a distance between the origin and the selection object. For example, when each line segment constituting the dotted line has a constant distance value, since the number of line segments constituting the dotted line varies according to the distance between the origin and the selection target, the distance can be intuitively grasped.

For example, the dotted line 650 of FIG. 13A and the dotted line 650 of FIG. 13B have the same length (the length between the first icon and the third icon), but the smartphone in the user space and the red rabbit 450 as a selection object ), since the number of line segments in FIG. 13A is large (dense) and the number of line segments is small in FIG. 13B, which is close in distance (because it is far), it is possible to determine the distance of the selection object from the smartphone.

14 is a flowchart of a method for providing a user interface further including a resizing step in response to a user's size change request according to an embodiment of the present invention. Referring to FIG. 14 , in one embodiment, the method further includes changing the size of the selected object according to the user's request for size change ( S2340 ). In one embodiment, the size change request includes all inputs for changing the size of the selection object of the user that can be input to the user client device, and the size of the selection object is in the user space 20, which is a virtual space. It may mean the size of the existing virtual selection object 450 .

For example, referring to FIGS. 15A and 15B , according to a size change request of the user of FIG. 15A , in one embodiment, the size of the red rabbit 450 as a selection target in FIG. 15B may be changed.

15A and 15B are diagrams for explaining a size change request by a user pinch-in/out according to an embodiment of the present invention. 15A and 15B , in an embodiment, the size change request includes a user's pinch in/out gesture. In an embodiment, the pinch-in/out gesture includes input of all pinch-in/out gestures of the user on the screen 382 of the client device while the selection object is selected.

In an embodiment, the pinch-in/out gesture for the size change request may mean a pinch-in/out gesture for an area adjacent to the selection object. For example, the size change request does not include a case in which the user inputs a pinch-in/out gesture to a screen area spaced apart from the selection object, but only a case in which the selection object is input around an output area. can

In an embodiment, the output size of the selection object may be differently output on the screen of the client device according to the change in the distance. For example, when there is a request to enlarge the size of the red rabbit 450 that is the selection object in FIG. 15A , the client device not only enlarges the size of the red rabbit existing in the user space 20 , but also the screen 382 of the client device ), the output size of the red rabbit can be enlarged as well.

In an embodiment, when the selection object is selected, an object indicating the size of the selection object may be further output. For example, referring to FIG. 15A , the size display object may display the size of a red rabbit 450 as a selection target as a numerical value 660 .

In an embodiment, the numerical value may indicate the size of the selected object in a specific unit (eg, 30 cm) or a magnification of the reference size (eg, 1.18x). In an embodiment, the reference size may mean a size when a specific object is initially uploaded to the user space or a preset size for each object.

13A and 13B and 15A and 15B , in an embodiment, the user may use the distance display objects 640 and 650 and the size display object 660 to determine the distance between the selection object and the origin and the selection object In addition to being able to easily grasp the size of , when the output size of the selected object on the screen of the client device is changed, there is an effect that the distance change and the size change can be distinguished. For example, when the output size is enlarged, the number of line segments or the distance value 640 of the dotted line 650 between the first icon and the third icon is maintained, and when the size value 660 is changed, the size is changed. In contrast to , when the size value is maintained but the number of line segments of the dotted line or the distance value is changed, it can be recognized as a distance change.

16 is a flowchart of a method for providing a user interface further comprising a rotation step according to a user's rotation request according to an embodiment of the present invention. Referring to FIG. 16 , in one embodiment, rotating the selection object according to the user's rotation request (S2360); further includes. In an embodiment, the rotation request includes all inputs for rotation of the user's selection object that may be input to the user client device.

In one embodiment, the rotation comprises a roll rotation, a yaw rotation or a pitch rotation.

17A and 17B are diagrams for explaining a roll rotation of a selection target according to an embodiment of the present invention. 17A and 17B , in an embodiment, the user's roll rotation request includes a rotate gesture. For example, when the user inputs a rotate gesture (384 in FIGS. 17A and 17B ) using two fingers as in FIGS. 17A and 17B , the direction of the rotation gesture (clockwise in FIGS. 17A and 17B ) direction), the selection object 450 may roll-rotate.

In an embodiment, the rotation gesture for the roll rotation request may refer to a rotation gesture for an area adjacent to the selection object. For example, the roll rotation request does not include a case in which the user inputs a rotation gesture to a screen area spaced apart from the selection object, but may mean only a case in which the selection object is input around an output area.

18A and 18B are diagrams for explaining yaw rotation of a selection target according to an embodiment of the present invention. 18A and 18B , in an embodiment, the user's yaw rotation request includes a pan or swipe gesture in a left and right direction. For example, when the user inputs a pan or swipe gesture ( 384 in FIGS. 18A and 18B ) in the right direction as shown in FIGS. 18A and 19B , according to the direction of the rotation gesture, the The selection object 450 may rotate yaw.

In an embodiment, the rotation gesture for requesting the yaw rotation may mean a pan or swipe gesture in a left and right direction for an area adjacent to the selection object. For example, the yaw rotation request does not include a case in which the user inputs a pan or swipe gesture in the left and right direction to the screen area at a location spaced apart from the selection object, but the selection object is output. It may mean only the case where input is made around the area.

19A and 19B are diagrams for explaining pitch rotation of a selection target according to an embodiment of the present invention. 19A and 19B , in an embodiment, the user's request for pitch rotation includes a pan or swipe gesture in the vertical direction. For example, when the user inputs a Pan or Swipe gesture ( 384 in FIGS. 19A and 19B ) in the upward direction as in FIGS. 19A and 19B , according to the direction of the rotation gesture, the The selection object 450 may rotate in pitch.

In an embodiment, the rotation gesture for the pitch rotation request may refer to a pan or swipe gesture in the vertical direction for an area adjacent to the selection object. For example, the yaw rotation request does not include a case in which the user inputs a pan or swipe gesture in the vertical direction to the screen area at a location spaced apart from the selection object, but the selection object is output. It may mean only the case where input is made around the area.

In an embodiment, the method may further include outputting voice information or haptic (eg, vibration) information corresponding to the completion of each operation when the distance is changed, the size is changed, or the rotation is rotated. For example, when a user requests a size change through a pinch-in/out gesture, when the selected object reaches a size desired by the user and the user stops the pinch-in/out gesture (when releasing the hand from the smartphone screen) ), a vibration indicating the completion of size change may be output.

In another embodiment, in the case of changing the distance, the voice information or haptic information may be output whenever the distance between the origin and the selection object is changed by a preset value. For example, if the distance between the origin and the selection object is 1 m and the distance is set to output vibration whenever the distance is changed by 10 cm, and the distance is changed according to the user's request for changing the distance, the distance is 110 cm, 120 cm, 130 cm Vibration can be output whenever it stretches to reach 90 cm, 80 cm, or 70 cm. Of course, the set value (10 cm) can be freely changed by the user.

In one embodiment, although not shown in the drawings, the method further includes: performing a change in the shape or color of the selection object in response to a user's request to change the shape or color of the selection object.

In one embodiment, although not shown in the drawings, the method further includes; uploading a specific object to the user space or changing the selected object to another object by the user. The object includes a computer graphic (CG) image of a specific object or an image of a real object or person, but is not limited thereto, and includes all objects that can be placed in a virtual space.

For example, in one embodiment, the user may upload a CG image of a lion other than a rabbit provided by the virtual space forming program in the user space, or upload an image taken by the user and arrange it through selection. Alternatively, while the red rabbit is selected, the selected object may be changed to an image captured by the user, but the user uploading or changing the object is not limited thereto.

In one embodiment, although not shown in the drawing, the method further includes; transmitting the first generated data, which is information on the corresponding user space 20, to the server according to the storage request of the user 200 . In an embodiment, the first generation data includes object information or arrangement location information of one or more objects arranged in the first user space.

In an embodiment, the first generated data includes a thumbnail image, and according to a user's request for taking a thumbnail image, obtaining a thumbnail image of the corresponding user space 20 and transmitting it to the server; further comprising .

In one embodiment, the thumbnail image is an image representing the corresponding user space, and when the user reconnects to the stored user space, the thumbnail image is captured and reconnected at the location at the time of storage, so that the thumbnail image is viewed The user space displayed in the thumbnail image can be implemented on the front side of the connected user (the front side of the user client device).

In an embodiment, the user's request to store or take a thumbnail image includes all user inputs that may be input to the user's client device, including, but not limited to, touching a specific icon.

6 is a block diagram of a client device providing a user interface according to an embodiment of the present invention. Referring to FIG. 6 , the client device 300 providing the user interface according to an embodiment of the present invention described above includes a user's selection request for a specific object or arrangement of a user for a selection object selected by the user. an input unit 350 for receiving a request; a detection unit 360 for detecting the location, direction, or operation information of the client device; a control unit 370 for moving the selection object in the user space according to the movement of the client device or a user's manipulation, and arranging the selection object in the user space according to the arrangement request; and an output unit 380 for outputting, by the control unit, a user space screen in a direction the client device is oriented, and for outputting the selection object to be fixedly positioned in one area of the screen according to the inputted user's selection request. ); includes.

Also, in one embodiment, the apparatus for providing a user interface according to an embodiment of the present invention may further include a configuration or function suitable for executing the user interface providing method according to the embodiment of the present invention described above. For example, in an embodiment, the input unit may further receive a size change request or a distance change request as well as a user's selection request or arrangement request. That is, the method for providing a user interface according to an embodiment of the present invention described above may be equally applied to the apparatus for providing a user interface according to an embodiment of the present invention.

In addition, the user interface providing method according to an embodiment of the present invention described above may be implemented as a program (or application) to be executed in combination with a computer, which is hardware, and stored in a medium.

The steps of a method or algorithm described in relation to an embodiment of the present invention may be implemented directly in hardware, as a software module executed by hardware, or by a combination thereof. A software module may contain random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.

As mentioned above, although embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can realize that the present invention can be embodied in other specific forms without changing its technical spirit or essential features. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

Claims

A method for a client device to provide a user interface for virtual object placement, the method comprising:

providing a user space screen in a direction the client device is oriented;

outputting the selection object to be fixedly positioned on one area of the screen in response to a user's request for selection of a specific object;

When the client device is moved by the user while the selection object is selected, the user space screen is provided according to the direction the client device is oriented, and the selection object is fixedly positioned on the one area of the screen. outputting; and

arranging the selection object in the user space in response to a user's request for placement of the selection object;

The user space is a virtual space including one or more virtual objects arranged by a user.
According to claim 1,

The location of the selection target is changed on a sphere at a specific distance from the origin according to the movement of the client device,

The origin is a location of the client device in a state in which the selection object is selected or a user location calculated from the location of the client device.
According to claim 1,

The selection request includes the user touching a specific object on the screen or the user touching the second icon while the first icon is located on the specific object,

The first icon is fixed to one area of the screen,

The second icon is spaced apart from the first icon and is located in another area of the screen.
According to claim 1,

The arrangement request includes, in a state in which the selection object is selected, a user touching the screen.
3. The method of claim 2,

Changing the distance between the origin and the selection object according to a user's request for changing the distance; further comprising, a user interface providing method.
6. The method of claim 5,

The method of providing a user interface, wherein the request for changing the distance includes a user tap or drag a third icon located in an area of the screen while the selected object is selected.
7. The method of claim 6,

The step of outputting the selection object to the screen comprises:

and outputting the selection object to be fixedly positioned on one area of the screen, further outputting an object connecting the selection object and the third icon and indicating a distance between the origin and the selection object.
According to claim 1,

Changing the size of the selected object in response to a user's request for size change; further comprising, a method for providing a user interface.
According to claim 1,

In response to a user's rotation request, rotating the selection object; further comprising, a user interface providing method.
A client device that provides a user interface for virtual object placement, the client device comprising:

an input unit for receiving a user's selection request for a specific object or a user's placement request for a selection object selected by the user;

a detection unit for detecting the position, direction, or operation information of the client device;

a control unit that moves the selection object in the user space according to the movement of the client device or a user's manipulation, and arranges the selection object in the user space according to the arrangement request; and

an output unit for outputting, by the control unit, a user space screen in a direction that the client device is oriented to, and outputting the selection object to be fixed and positioned in one area of the screen according to the inputted user's selection request; and,

The user space is a virtual space including one or more objects placed by a user, the client device.
A program for providing a user interface for object placement, which is combined with a computer as hardware and stored in a medium to execute the method of claim 1 .