KR20190125567A - Method and system for providing contents based on story using virtual reality and augmented reality - Google Patents

Abstract

The present invention relates to a method and a system for providing content based on a story using virtual reality and augmented reality. Content is generated according to a selection input of a user, and the generated content can be provided as an augmented reality service and a virtual reality service according to a surrounding environment at a location of the user.

Description

METHOD AND SYSTEM FOR PROVIDING CONTENTS BASED ON STORY USING VIRTUAL REALITY AND AUGMENTED REALITY}

The present invention relates to a method and system for providing story-based content using augmented reality / virtual reality, and more particularly, to provide content generated by a user's selection input to an augmented reality service and a virtual reality service according to a surrounding environment. will be.

Augmented Reality (AR) is a space where physical space and electronic space are fused together and adds virtual information to the user's perception by mixing virtual images such as computer graphics with real or real world images. Used to reinforce. In addition, VR (Virtual Reality) allows the user to recognize and experience the virtual world created artificially using special headsets or peripherals as if they were real.

Recently, due to the development of smart phones, it is possible to make drawings of character shapes using applications, create coloring and comments, and make cartoons, animations, children's fairy tales, writing, diaries, photo books, etc. Existing applications gave users satisfaction by providing their own storybooks (or comic books) that woven comics.

However, existing technologies cannot provide new contents using augmented reality and virtual reality technology and storybooks, and there is no way to apply a comic book or storybook made by the user to augmented reality and virtual reality. to be.

An object of the present invention is to generate content according to a user's selection input, and provide the generated content as an augmented reality service and a virtual reality service according to the surrounding environment at the user location.

In the method of operating a content providing system that is an application installed on a mobile device possessed by a user according to an embodiment of the present invention, generating content including a character-shaped picture based on a user's selection input, Photographing the surrounding environment at the user location and providing the augmented reality service or the virtual reality service by overlapping the generated content with the photographed surrounding environment.

The generating of the content may generate the content including at least one of a picture, a color, and a comment according to a user's selection input, and may modify, delete, or move the content.

The generating of the content may include generating a story-based storybook using the content generated in each of the plurality of pages, and generating the content included in the storybook in an animation, image, or multimedia form.

The providing of the augmented reality service or the virtual reality service may provide the virtual reality service that recognizes the content through the augmented reality service and a peripheral device that overlap the content according to the surrounding environment at the user location. .

The providing of the augmented reality service or the virtual reality service may provide specific content at a specific location to the augmented reality service or the virtual reality service, based on the user location or data signal detected through a sensor.

The providing of the augmented reality service or the virtual reality service may provide the specific content different according to a surrounding environment, a path, and an area based on the user location, and animate a story-based storybook using the specific content. It can be provided in the form of images or multimedia.

The providing of the augmented reality service or the virtual reality service may provide the augmented reality service or the virtual reality service in a state where the user stops, and stop or temporarily stop the augmented reality service or the virtual reality service according to a user's movement. You can stop, pause or delete it.

The providing of the augmented reality service or the virtual reality service may provide at least one or more effects of expanding, reducing, rotating, moving, dragging, or a combination thereof according to a user's selection input.

In the content providing system, which is an application installed on a mobile device possessed by a user according to an embodiment of the present invention, a content generation unit for generating content including a figure of a character shape based on a user's selection input; And a photographing unit for photographing a surrounding environment at a location, and a content providing unit for providing an augmented reality service or a virtual reality service by overlapping the generated content with the photographed surrounding environment.

In addition, the content providing system is an application (Application) installed in the mobile device possessed by the user according to an embodiment of the present invention is a position sensor for detecting the user location and movement, a signal detector for detecting a data signal at a specific location And a controller configured to control the content generated by the content generator and external content received from the outside to the augmented reality service or the virtual reality service through the content provider.

The content generator may generate the content including at least one of a picture, a color, and a comment according to a user's selection input, and may modify, delete, or move the content.

The content unit may generate a story-based storybook using the content generated in each of the plurality of pages, and generate the content included in the storybook in the form of animation, image, or multimedia.

The content provider may provide the virtual reality service that recognizes the content through the augmented reality service and a peripheral device that overlap the content according to the surrounding environment at the user location.

The content provider may provide specific content at a specific location to the augmented reality service or the virtual reality service based on the user location or data signal detected by the location sensor and the signal detector.

The content providing unit may provide the specific content different according to a surrounding environment, a path, and a region based on the user's location, and may provide a story-based storybook in the form of animation, image, or multimedia using the specific content. .

The content providing unit provides the augmented reality service or the virtual reality service while the user is stopped, and may stop, pause, temporarily delete or delete the augmented reality service or the virtual reality service according to a user's movement.

The content providing unit may provide at least one effect of expanding, reducing, rotating, moving, dragging, or a combination thereof according to a user's selection input.

According to an embodiment of the present invention, the content can be generated according to a user's taste, and the content generated by the user can be applied to the surrounding environment to experience in the form of augmented reality service and virtual reality service through a mobile device. .

1 is a flowchart illustrating a story-based content providing method applying augmented reality / virtual reality according to an embodiment of the present invention.
2 illustrates an example of generating content according to a user's selection input according to an embodiment of the present invention.
3 to 8 illustrate an example of providing an augmented reality service at a user location according to an embodiment of the present invention.
9 illustrates an example of providing a virtual reality service at a user location according to an embodiment of the present invention.
10 is a block diagram illustrating a configuration of a story-based content providing system to which augmented reality / virtual reality is applied according to an embodiment of the present invention.
11 is a view illustrating a detailed configuration of a story-based content providing system applying augmented reality / virtual reality according to an embodiment of the present invention.
12 shows an example of a VR product.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited or limited by the embodiments. Also, like reference numerals in the drawings denote like elements.

Also, the terminology used herein is a term used to properly express a preferred embodiment of the present invention, which may vary depending on a viewer, an operator's intention, or customs in the field to which the present invention belongs. Therefore, the definitions of the terms should be made based on the contents throughout the specification.

Embodiments of the present invention is to provide the content generated by the user's selection input to the augmented reality service or virtual reality service.

Story-based content providing method and system applying augmented reality / virtual reality according to an embodiment of the present invention may be an application (Application) installed on the mobile device possessed by the user, generates the content according to the user's selection input, The augmented reality service or the virtual reality service may be provided by overlapping the generated content and the surrounding environment at the user location.

A mobile device possessed by a user refers to a device that can transmit and receive various data according to a user's key manipulation or command, and provides augmented reality and virtual reality services. According to an embodiment, the mobile device may be a tablet PC, a laptop, a personal computer, a smart phone, a personal digital assistant, or a mobile communication. The terminal may be one of a mobile communication terminal and a notebook.

Furthermore, the mobile device refers to all terminal devices capable of accessing a web / mobile site or installing and executing a service-only application associated with a story-based content providing method and system applying augmented reality / virtual reality according to an embodiment of the present invention. can do. In this case, the mobile device may perform overall service operations such as service screen configuration, data input, data transmission, and data storage under the control of a web / mobile site or a dedicated application.

Hereinafter, the present invention will be described in more detail with reference to FIGS. 1 to 11.

1 is a flowchart illustrating a story-based content providing method applying augmented reality / virtual reality according to an embodiment of the present invention.

The method of FIG. 1 may be performed by a story-based content providing system applying augmented reality / virtual reality according to an embodiment of the present invention shown in FIG. 10.

Referring to FIG. 1, in step 110, content including a figure of a character shape is generated based on a user's selection input.

For example, a user may draw a figure of a character through an application installed in a mobile device possessed by the user, and may paint a color or write a comment on the figure of the figure. Accordingly, step 110 may be a step of generating content including at least one or more of a picture, a color, and a comment according to a user's selection input, and performing editing such as editing, deleting, or moving the content.

In operation 110, a story-based storybook may be generated using content generated in each of the plurality of pages, and content included in the storybook may be generated in an animation, image, or multimedia form. For example, step 110 may be provided in the form of a preset story-based animation using the generated content, wherein the background of the animation is configured to reflect the augmented reality and virtual reality or be configured with a preset animation Can be.

Accordingly, the content described in the present invention may include a picture, a color, and a comment generated by a user's selection input, and may refer to both an animation, an image, or a multimedia form.

In step 120, the environment at the user location is photographed.

Operation 120 may be an operation of acquiring actual image information about the surrounding environment captured by a camera included in the mobile device possessed by the user. In operation 120, actual image information may be obtained using video streaming of a camera. In this case, the camera may be implemented in various forms as long as it supports image precision and acquisition speed which can accurately acquire real image information in real time.

According to an embodiment, step 120 may acquire real-time image information captured by the camera, or may obtain image information that is pre-stored in units of seconds and minutes in the camera or the memory unit of the present invention. That is, the method and system for providing story-based content applying augmented reality / virtual reality according to an embodiment of the present invention may include at least one or more of image information obtained in real time and pre-stored image information according to an embodiment or environment to which the present invention is applied. Can be used. In this case, the image information may be obtained by a camera or a memory unit of the present invention, or may be obtained from an external server.

In operation 130, the augmented reality service or the virtual reality service is provided by overlapping the generated content with the photographed surrounding environment.

In operation 130, the augmented reality service overlapping the content according to the surrounding environment at the user location and the virtual reality service recognizing the content through the peripheral device may be provided.

For example, the mobile device possessed by the user receives current location information from a GPS satellite, or detects the current location information through at least one sensor of an acceleration sensor, a gyro sensor, a proximity sensor, and a motion sensor. Based on the augmented reality development tool (for example, Apple development tool ARKit or Google development tool ARcore) based on the augmented reality location information, such as the moving angle, azimuth, radius, etc. based on the mobile device location information, direction information And augmented reality location-related information such as search range information. In addition, the mobile device possessed by the user may be installed in a peripheral device such as a head mounted display (HMD), and a lens is provided inside the peripheral device to divide the screen output from the mobile device to the left and right, respectively. The image may be entered into one eye, that is, the image may be left and right, respectively, so that the user may feel in three dimensions.

At this time, the peripheral device may be applied to a cardboard, a VR product similar to the cardboard function or a portable product as shown in Figure 12, the configuration and principle of the cardboard is a known technique, detailed description thereof will be omitted.

In this way, step 130 may provide the content generated by the user's selection input in the form of an augmented reality service or a virtual reality service.

In operation 130, a specific content at a specific location may be provided as an augmented reality service or a virtual reality service based on a user location or a data signal sensed by the sensor. More specifically, step 130 provides different specific content according to the surrounding environment, path, and zone based on the user's location, and provides a story-based storybook in the form of animation, image, or multimedia using the specific content. can do.

In other words, step 130 may be provided in the form of a moving animation or multimedia using a content generated by a user's selection input or a specific content at a specific location, or may be provided in the form of a moving image or multimedia.

Story-based content providing method and system applying augmented reality / virtual reality according to an embodiment of the present invention in addition to the content generated according to the user's selection input, augmented reality service or virtual reality content received from the outside or pre-stored in the application It may be provided in the form of a service, which may be specific content set at a specific location. For example, if the user is located in the Han River, step 130 may provide dinosaur content moving in a large lawn from the user's location in the form of augmented reality service or virtual reality service. As another example, when the user is located in Bulguksa, step 130 may provide content related to Bulguksa history and the background of the era in the form of an augmented reality service or a virtual reality service.

Furthermore, the method and system for providing story-based content using augmented reality / virtual reality according to an embodiment of the present invention is based on the geographical location, route, and zone information of the specific location. Different specific contents may be provided in the form of augmented reality service or virtual reality service according to the path, area, and zone.

In this case, the provided content or specific content may be based on a combination of at least one of content generated by the user, external content received from the outside, and predetermined content.

In operation 130, the augmented reality service or the virtual reality service may be provided while the user is stopped, and the augmented reality service or the virtual reality service may be stopped, paused, temporarily deleted, or deleted according to the movement of the user.

For example, assuming that a user experiences augmented reality service or virtual reality service through a mobile device in the Han River, a user who concentrates on the experience of a mobile device may not be able to secure a vision, which may cause a dangerous situation. Accordingly, step 130 may provide an augmented reality service or a virtual reality service only when the user is stationary, and may stop, pause, temporarily delete or delete content when a user's movement occurs.

However, the stop, pause, temporary deletion or deletion of the content may be set by a user's selection input, and augmented reality / virtual reality according to an embodiment of the present invention based on a field of view securement rate and risk possibility based on the user's location The story-based content providing method and system may be set in advance.

In operation 130, the augmented reality service may be provided with at least one or more effects of expanding, contracting, rotating, moving, dragging, or a combination thereof.

For example, operation 130 may be performed by the user touching the display of the mobile device to rotate, move, enlarge, reduce, drag, and combine the contents. The control includes all operations of changing the motion, size, color, shape, shape, and the like of the content.

Story-based content providing method and system applying augmented reality / virtual reality according to an embodiment of the present invention augmented the illustrations or pictures inserted in the teaching materials, such as language teaching materials, encyclopedias, science texts, medical materials with voice explanations It can be shown as a reality service or a virtual reality service, and can have an excellent educational effect. In addition, public data such as traffic and streets and data such as buildings and locations using location information may be recognized and represented as augmented reality services or virtual reality services. In addition, it allows customers to indirect experiences at events, large conferences, performances, etc., and can recognize historical buildings and express historical facts with augmented reality services or virtual reality services.

2 illustrates an example of generating content according to a user's selection input according to an embodiment of the present invention.

Referring to FIG. 2, a method and system for providing story-based content applying augmented reality / virtual reality according to an embodiment of the present invention includes content including at least one of a picture, a color, and a comment according to a user's selection input. 210 may be generated and the content 210 may be edited, deleted, moved, or the like.

Referring to FIG. 2A, a user may generate a content 210 which is a picture of a character shape by touching a display of a mobile device, and may color the content 210 or edit size, position, and the like. have. In addition, as shown in FIG. 2B, the user may add a comment on the content 210.

For this reason, referring to FIG. 2 (c), a method and system for providing story-based content applying augmented reality / virtual reality according to an embodiment of the present invention includes at least one of a picture, a picture, a color, and a comment. The content 210 may be generated, and story-based animation, images, or multimedia may be generated using the generated content 210.

3 to 8 illustrate an example of providing an augmented reality service at a user location according to an embodiment of the present invention.

Referring to FIG. 3, the user activates a story-based content providing method and system 400 to which augmented reality / virtual reality is applied according to an embodiment of the present invention through a mobile device, and includes a surrounding appendix 300. In the case of photographing, the story-based content providing method and system 400 applying augmented reality / virtual reality according to an embodiment of the present invention recognizes the location of the user.

At this time, the story-based content providing method and system 400 applying augmented reality / virtual reality according to an embodiment of the present invention from the current position information received from the GPS satellite or the current position information through the sensor included in the mobile device 4, a data signal received from a terminal device 310 such as a beacon or near field communication (NFC) disposed in a surrounding environment at the user location, as shown in FIG. 4. It is possible to detect the user location based on.

5 and 6, a story-based content providing method and system 400 using augmented reality / virtual reality according to an embodiment of the present invention corresponds to a surrounding environment 300 including a corpus according to a user's location. The predetermined first specific content 510 and the second specific content 520 may be provided as an AR service through a mobile device.

For example, when the user photographs the surrounding environment 300 including the high peak through the mobile device, the first specific content 510 and the second specific content 520 are displayed in the surrounding environment 300 including the high peak. ) Is overlapped so that you can experience augmented reality service.

At this time, the story-based content providing method and system 400 applying augmented reality / virtual reality according to an embodiment of the present invention, to provide a variety of different content in the form of augmented reality service according to the user's eyes and movements. Can be. For example, when the user moves the mobile device to the right from FIG. 5 to FIG. 6, not only the first specific content 510 representing the background of the existing grass blades but also the second specific content 520 representing the dinosaur are newly added. May appear.

Furthermore, when the user moves up the mobile device in FIG. 6, it can be seen that the second specific content 520, which only represents the dinosaur's leg in FIG. 6, also represents the head of the dinosaur in FIG. 7 as the mobile device moves. have.

That is, when the user moves the position of the mobile device up, down, left, and right, the story-based content providing method and system 400 applying augmented reality / virtual reality according to an embodiment of the present invention is further modified according to the movement. Various contents can be provided, and a user can experience various contents.

Referring to FIG. 8, a story-based content providing method and system 400 using augmented reality / virtual reality according to an exemplary embodiment of the present invention may be preset in response to a surrounding environment 300 including a peak according to a user's location. The generated content 530 may be provided as an augmented reality service through a mobile device.

For example, the generated content 530 is content generated by a user's selection input, and the user draws a picture of the character shape through the mobile device, colors the picture, and writes a comment on the picture of the character shape. It may be generated content.

In this case, the specific location where the generated content 530 is provided may be preset according to a user's preference, or may be provided by a user's call at the user's location.

9 illustrates an example of providing a virtual reality service at a user location according to an embodiment of the present invention.

Referring to FIG. 9, a story-based content providing method and system 600 to which augmented reality / virtual reality is applied according to an exemplary embodiment of the present invention may be preset in response to a surrounding environment 300 including a peak point according to a user's location. The first specific content 510 and the generated content 530 may be provided as a virtual reality service through a peripheral device.

The story-based content providing method and system 600 applying augmented reality / virtual reality according to an embodiment of the present invention may provide a virtual reality service through a peripheral device such as a head mounted display (HMD). . As shown in FIG. 9, the mobile device is located in front of the peripheral device, and the peripheral device has a lens inside to divide the screen output from the mobile device from side to side. Each can come in to the right to make the user feel in three dimensions.

Contents 510, 520, and 530 illustrated in FIGS. 5 to 9 are content generated by a user, content received from the outside, and a story-based content providing method using augmented reality / virtual reality according to an embodiment of the present invention. And it may be a combination of at least one or more of the content previously stored in the system 400, may be a picture, color, comment of the shape of the character, may be story-based animation, image or multimedia. Furthermore, the number, location, and shape of the contents 510, 520, and 530 are not limited to those shown in the drawings.

10 is a block diagram illustrating a configuration of a story-based content providing system to which augmented reality / virtual reality is applied according to an embodiment of the present invention.

Referring to FIG. 10, the story-based content providing system 1000 to which augmented reality / virtual reality is applied according to an embodiment of the present invention provides augmented reality service and virtual reality according to a surrounding environment. Provided as a service.

To this end, the story-based content providing system 1000 to which augmented reality / virtual reality is applied according to an embodiment of the present invention includes a content generating unit 1010, a photographing unit 1020, and a content providing unit 1030. In addition, the story-based content providing system 1000 to which augmented reality / virtual reality is applied according to an embodiment of the present invention may further include a location sensor 1040, a signal detector 1050, and a controller 1060. .

The content generator 1010 generates a content including a picture of a character shape based on a user's selection input.

For example, a user may draw a figure of a character through an application installed in a mobile device possessed by the user, and may paint a color or write a comment on the figure of the figure. Accordingly, the content generator 1010 may generate content including at least one of a picture, a color, and a comment according to a user's selection input, and may edit, delete, or move the content.

In addition, the content generator 1010 may generate a story-based storybook using the contents generated in each of the plurality of pages, and may generate the content included in the storybook in the form of animation, image, or multimedia. For example, the content generator 1010 may provide a preset story-based animation form using the generated content, wherein the background of the animation is configured or preset to reflect augmented reality and virtual reality. It can be composed of animations.

Accordingly, the content described in the present invention may include a picture, a color, and a comment generated by a user's selection input, and may refer to both an animation, an image, or a multimedia form.

The photographing unit 1020 photographs a surrounding environment at a user location.

The photographing unit 1020 may obtain actual image information about the surrounding environment photographed from a camera provided in the mobile device possessed by the user. In this case, the photographing unit 1020 may acquire the actual image information by using the video streaming of the camera, and the camera may be implemented in various forms if it supports the image precision and the acquisition speed to obtain the actual image information accurately in real time. This is possible.

According to an exemplary embodiment, the photographing unit 1020 may obtain real-time image information captured by the camera, or may obtain image information previously stored in seconds or minutes in the camera or the memory unit 1120 of the present invention. That is, the method and system for providing story-based content applying augmented reality / virtual reality according to an embodiment of the present invention may include at least one or more of image information obtained in real time and pre-stored image information according to an embodiment or environment to which the present invention is applied. Can be used. In this case, the image information may be obtained by the camera or the memory unit 1020 of the present invention, or may be obtained from an external server.

The location sensor 1040 may detect a user's location and movement. For example, the location sensor 1040 receives the current location information of the mobile device possessed by the user from the GPS satellite, or through at least one of an acceleration sensor, a gyro sensor, a proximity sensor, and a motion sensor of the mobile device. Current location information can be detected. Thus, the story-based content providing system 1000 applying augmented reality / virtual reality according to an embodiment of the present invention is based on the current position information augmented reality development tool (for example, Apple development tool ARKit or Google development tool ARcore By using augmented reality position information, such as the moving angle, azimuth, radius, etc. obtained using a) may be generated augmented reality position-related information such as position information, direction information, search range information of the mobile device.

In addition, the mobile device possessed by the user may be installed in a peripheral device such as a head mounted display (HMD), and a lens is provided inside the peripheral device to divide the screen output from the mobile device to the left and right, respectively. The image may be entered into one eye, that is, the image may be left and right, respectively, so that the user may feel in three dimensions.

The signal detector 1050 may detect a data signal at a specific position. For example, the signal detector 1050 may detect a data signal received from a terminal device such as a beacon or a near field communication (NFC) disposed in a surrounding environment at a user location. The data signal may include specific location information and information related to specific content set at a specific location.

In addition, the signal detector 1050 may detect a data signal at a specific location of the user through at least one sensor among an acceleration sensor, a gyro sensor, a proximity sensor, and a motion sensor included in the mobile device. For example, the signal detector 1050 may receive and connect the closest data signal among the data signals received from the plurality of terminal devices according to the location of the user sensed by the location detector 1040.

The content provider 1030 overlaps the generated content with the photographed surrounding environment and provides an augmented reality service or a virtual reality service.

The content provider 1030 may provide an augmented reality service that overlaps content according to a surrounding environment at a user location, and a virtual reality service that recognizes content through a peripheral device.

The content provider 1030 may provide specific content at a specific location as an augmented reality service or a virtual reality service, based on a user location or data signal detected by the location sensor 1040 and the signal detector 1050. Can be. In more detail, the content providing unit 1030 provides different specific content according to the surrounding environment, path, and zone based on the user's location, and animates, images, or creates a storybook based on the specific content. It can be provided in a multimedia form.

Story-based content providing system 1000 applying augmented reality / virtual reality according to an embodiment of the present invention, in addition to the content generated according to the user's selection input, augmented reality service or virtually stored content received from the outside or pre-stored in the application It may be provided in the form of a reality service, which may be specific content set at a specific location. For example, when the user is located in the Han River, the content provider 1030 may provide dinosaur content moving in a large lawn based on the location of the user in the form of augmented reality service or virtual reality service. As another example, when the user is located in Bulguksa, step 130 may provide content related to Bulguksa history and the background of the era in the form of an augmented reality service or a virtual reality service.

Furthermore, the story-based content providing system 1000 to which augmented reality / virtual reality is applied according to an embodiment of the present invention may be located at the same location using information such as geographical surroundings, routes, and zones of the specific location. Depending on the environment, path, and area, different specific content may be provided in the form of augmented reality service or virtual reality service.

In this case, the provided content or specific content may be based on a combination of at least one of content generated by the user, external content received from the outside, and preset content.

The content provider 1030 provides an augmented reality service or a virtual reality service in a state where the user stops, and may stop, pause, temporarily delete or delete the augmented reality service or the virtual reality service according to the user's movement.

For example, assuming that a user experiences augmented reality service or virtual reality service through a mobile device in the Han River, a user who concentrates on the experience of a mobile device may not be able to secure a vision, which may cause a dangerous situation. Accordingly, the content providing unit 1030 provides augmented reality service or virtual reality service only when the user is stationary, and when the user's movement occurs, the content providing unit 1030 may stop, pause, temporarily delete or delete the content.

However, the stop, pause, temporary deletion or deletion of the content may be set by a user's selection input, and augmented reality / virtual reality according to an embodiment of the present invention based on a field of view securement rate and risk possibility based on the user's location. The story-based content providing system 1000 may be set in advance.

The content provider 1030 may provide at least one or more effects of expanding, reducing, rotating, moving, dragging, or a combination thereof, according to a user's selection input.

For example, the content provider 1030 may control the user by rotating, moving, enlarging, reducing, dragging, and combining the contents by touching the display of the mobile device. The control includes all operations of changing the motion, size, color, shape, shape, and the like of the content.

The controller 1060 may control to provide the content generated by the content generator 1010 and the external content received from the outside as an augmented reality service or a virtual reality service through the content provider 1030.

11 is a view illustrating a detailed configuration of a story-based content providing system applying augmented reality / virtual reality according to an embodiment of the present invention.

11 is only an example of a story-based content providing system 1000 applying augmented reality / virtual reality, and a story-based content providing system 1000 applying augmented reality / virtual reality is configured with more or less components than those shown in FIG. It may include elements, combine two or more components, or include configurations or arrangements that include different components.

In addition, the components shown in FIG. 11 may be implemented in hardware, software, or a combination of both hardware and software, including integrated circuits specialized for one or more signal processing or applications. The term 'herein' refers to a unit for processing at least one function or operation, which may be implemented by hardware or software, or a combination of hardware and software.

Referring to FIG. 11, the story-based content providing system 1000 applying augmented reality / virtual reality according to an embodiment of the present invention includes a display unit 1110, a memory unit 1120, a communication unit 1130, and an input unit 1140. ), An output unit 1150 and a control unit 1060.

The display 1110 may display a screen including a plurality of items positioned in each of the plurality of regions, and may receive a touch input from a user based on a haptic contact or tactile contact. Another screen may be displayed that includes at least one or more items related to the function based on the sensing surface, the sensor, or the sensor set. In addition, the display 1110 may provide a user interface between the story-based content providing system 1000 to which augmented reality / virtual reality is applied according to an embodiment of the present invention.

The memory unit 1120 may include a high-speed random access memory, a magnetic disk, an SRAM, a DRAM, a ROM, a flash memory, or a nonvolatile memory.

The memory unit 1120 may include a software module, an instruction set, or various other data necessary for the operation of the story-based content providing system 1000 to which augmented reality / virtual reality is applied according to an exemplary embodiment of the present invention. In one embodiment, a storybook including a resource, an image, and a plurality of pages that are activated according to a user's selection input for generating content may be stored and maintained, which is stored as a customized history for each of the plurality of users. Can be maintained.

In addition, access to the memory unit 1120 may be controlled by other components such as the controller 1060 and the peripheral interface.

The communication unit 1130 may transmit and receive an RF signal, also known as an electromagnetic signal. In addition, the communication unit 1130 may convert an electrical signal into an electromagnetic signal or convert an electrical signal from the electromagnetic signal, and communicate with a communication network, another communication device, and an external device through the electromagnetic signal.

In one embodiment, the communication unit 1130 may include circuitry for performing this function, the circuitry comprising an antenna system, an RF transceiver, one or more amplifiers, tuners, one or more oscillators, digital signal processors, CODEC chipsets. , Subscriber identity module (SIM) card, memory, and the like, but are not limited thereto.

In an embodiment, the communication unit 1130 may be a wireless network such as a cellular telephone network, a wireless local area network (LAN) and / or a metropolitan area network (MAN), a network such as an intranet and / or the Internet, also called the World Wide Web (WWW). And other devices by wireless communication.

Such wireless communications include Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), wideband code division multiple access (W-CDMA), code division multiple access (CDMA), time division multiple access (TDMA), and Bluetooth ( Bluetooth), Wireless Fidelity (Wi-Fi) (such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and / or IEEE 802.11n), voice over Internet Protocol (VoIP), Wi-MAX, Long Term Evolution (LTE) Protocols for email, such as Zigbee, Z-wave, Bluetooth Low Energy (BLE), Beacon, Internet Message Access Protocol (IMAP) and / or Post Office Protocol (POP), eXtensible Messaging and Presence Protocols, Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS), or Short Message Service (SMS), or communication protocols not developed at the time of filing of the present application. It may include. However, a plurality of communication standards, protocols, and technologies, which are not limited thereto, may be used in the above-described wireless communication.

For example, the communicator 1130 may communicate with an external server to call and provide external content.

The input unit 1140 may include a keyboard, a touch pad, a dial, a slider switch, a joystick, and the like, and may be combined with any of pointer devices such as an infrared port, a USB port, and a mouse.

The output unit 1150 may include an audio module, a speaker module, and a vibration module.

In one embodiment, the output unit 1150 may output alarm information that may occur during the production and editing of the content.

According to an embodiment, the output unit 1150 may output a warning sound by using an audio module, a speaker module, or a buzzer, or may flash a warning light by using a light emitting diode (LED), and output vibration by using a vibration module. can do.

The controller 1060 may include a processor, a central processing unit, an application processor, or a communication processor. For example, the controller 1060 may be augmented reality according to an embodiment of the present invention performed by the display 1110, the memory 1120, the communication unit 1130, the input unit 1140, and the output unit 1150. The operation of the story-based content providing system 1000 to which virtual reality is applied may be controlled.

In one embodiment, the control unit 1060 is an operation related to the control and / or communication of at least one other component of the story-based content providing system 1000 applying augmented reality / virtual reality according to an embodiment of the present invention or Data processing can be performed.

In an embodiment, the controller 1060 may control a plurality of hardware or software components connected to the controller 1060 by driving an operating system or an application program, and may perform various data processing and operations.

For example, the controller 1060 may be implemented as a system on chip (SOC), and the controller 616 may transmit a command or data received from at least one of other components (eg, nonvolatile memory) to the volatile memory. It can be loaded and processed, and the resulting data stored in nonvolatile memory.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors, microcomputers, field programmable arrays (FPAs), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of explanation, one processing device may be described as being used, but one of ordinary skill in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process independently or collectively. You can command the device. Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. Or may be permanently or temporarily embodied in a signal wave to be transmitted. The software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.

The method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components Or even if replaced or substituted by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims that follow.

Claims (18)

In the operation method of the content providing system which is an application (Application) installed in the mobile device possessed by the user,
Generating content including a character-shaped picture based on a user's selection input;
Photographing the surrounding environment at the user location; And
Providing an augmented reality service or a virtual reality service by overlapping the generated content with the photographed surrounding environment;
Story-based content providing method applying augmented reality / virtual reality comprising a. The method of claim 1,
Generating the content
A story-based content providing method of applying the augmented reality / virtual reality to generate the content including at least one or more of the picture, color and comments according to the user's selection input, and to modify, delete, and move the content. The method of claim 2,
Generating the content
Story based content using story content based on the contents generated in each of a plurality of pages, and augmented reality / virtual reality applied to augmented reality / virtual reality for generating the content included in the storybook in the form of animation, image or multimedia. How to Provide. The method of claim 1,
Providing the augmented reality service or virtual reality service is
And augmented reality / virtual reality providing story-based content providing the virtual reality service for recognizing the content through the augmented reality service and a peripheral device overlapping the content according to the surrounding environment at the user location. The method of claim 4, wherein
Providing the augmented reality service or virtual reality service is
Story-based content providing method applying augmented reality / virtual reality to provide a specific content at a specific location to the augmented reality service or the virtual reality service based on the user location or data signal detected by the sensor. The method of claim 5,
Providing the augmented reality service or virtual reality service is
The augmented reality / virtual reality that provides the specific content different from the user's location according to the surrounding environment, the path, the zone, and provides the story-based storybook in the form of animation, image or multimedia using the specific content. Applied story-based content providing method. The method of claim 4, wherein
Providing the augmented reality service or virtual reality service is
Augmented reality, characterized in that for providing the augmented reality service or the virtual reality service in the user's stop, stop, pause, temporarily delete or delete the augmented reality service or the virtual reality service according to the user's movement Story-based content providing method using virtual reality. The method of claim 4, wherein
Providing the augmented reality service or virtual reality service is
Story-based content providing method applying augmented reality / virtual reality that provides at least one or more of the effect of expanding, reducing, rotating, moving, dragging, or a combination thereof. A computer program stored in a computer readable recording medium for performing the method of any one of claims 1 to 8. In the content providing system that is an application (Application) installed in the mobile device possessed by the user,
A content generation unit generating content including a picture of a character shape based on a user's selection input;
Photographing unit for photographing the surrounding environment at the user location; And
Content providing unit for providing an augmented reality service or virtual reality service by overlapping the generated content and the photographed surrounding environment
Story-based content providing system applying augmented reality / virtual reality, including. The method of claim 10,
A position detecting unit detecting the user position and movement;
A signal detector detecting a data signal at a specific position; And
Control unit for controlling to provide the content generated by the content generating unit and the external content received from the outside to the augmented reality service or the virtual reality service through the content providing unit
Story-based content providing system applying augmented reality / virtual reality further comprising a. The method of claim 10,
The content generation unit
A story-based content providing system that applies augmented reality / virtual reality that generates the content including at least one of a picture, a color, and a comment according to a user's selection input, and which modifies, deletes, and moves the content. The method of claim 12,
The content unit
Story based content using story content based on the contents generated in each of a plurality of pages, and augmented reality / virtual reality applied to augmented reality / virtual reality for generating the content included in the storybook in the form of animation, image or multimedia. Providing system. The method of claim 10,
The content providing unit
Story-based content providing system applying augmented reality / virtual reality to provide the virtual reality service for recognizing the content through the augmented reality service and the peripheral device overlapping the content according to the surrounding environment at the user location. The method of claim 11,
The content providing unit
Story-based application of augmented reality / virtual reality that provides specific content at a specific location to the augmented reality service or the virtual reality service based on the user position or data signal detected by the position sensor and the signal detector. Content providing system. The method of claim 15,
The content providing unit
The augmented reality / virtual reality that provides the specific content different from the user's location according to the surrounding environment, the path, the zone, and provides the story-based storybook in the form of animation, image or multimedia using the specific content. Applied story-based content providing system. The method of claim 11,
The content providing unit
Augmented reality, characterized in that for providing the augmented reality service or the virtual reality service in the user's stop, stop, pause, temporarily delete or delete the augmented reality service or the virtual reality service according to the user's movement Story-based content providing system that applies / virtual reality. The method of claim 10,
The content providing unit
Story-based content providing system applying augmented reality / virtual reality that provides at least one or more of the effects of expanding, reducing, rotating, moving, dragging, or a combination thereof.

Images