KR102096883B1 - Method and system for advertizing - Google Patents

Abstract

Disclosed is a method for providing advertisements and a system for the same. In the advertisement providing method, the system determines a display area identified by the user among 360 images, adding the display area determined by the system as an advertising area in which predetermined advertisement content can be placed, and the system is configured to And disposing predetermined advertisement content in the advertisement area.

Description

Advertisement provision method and system for performing it {Method and system for advertizing}

The present invention relates to a method and a system capable of generating a matched image, and more specifically, to generate a relatively accurate and effective matched image (for example, a 360 degree image) without specialized equipment using a user's terminal. It relates to a method and system that can be done.

In addition, the present invention is directed to a method and system capable of providing advertisements using a matched image or a 360 image.

Attempts to create a matched image to provide a virtual reality or highly realistic simulation environment have been widely used.

For example, the panoramic image may mean an image in which a plurality of images are combined horizontally (left and right) so that a horizontal viewing angle can cover 180 degrees to 360 degrees.

In addition, a 360-degree image may mean an image that can be covered both up, down, left, and right, centering on a user, and can generally be obtained by arranging several images in a sphere or a mecator.

However, in order to acquire such a panoramic image or a 360 degree image, professional equipment is required. Recently, a user wants to generate a matched image (for example, a panoramic image or a 360-degree image) by holding a mobile terminal (eg, a smart phone) and rotating the mobile terminal to acquire a plurality of photos and matching the acquired pictures. Attempts existed.

However, in this case, since the viewing angle or the shooting direction of the plurality of pictures is not correct, stitching of the image takes considerable time and the accuracy is also low.

In addition, conventionally, when performing the advertisement using the generated matched image or the 360 image, a method having no difference from the advertisement method used in the simple image has been used. That is, a method in which the characteristics of the 360 image is not reflected is used.

Therefore, there is a need for an advertising method suitable for the characteristics of a 360 image.

Korean Patent Application Publication No. 10-2002-0078663 "Digital engraving image registration processing method and matching processing device for positioning multiple pieces of images in a digital mosaic image configuration" Korean Patent Application Publication No. 10-2014-0147597 "Mobile terminal and control method therefor" Korean Patent Application Publication No. 10-2002-0026791 "Photo-based virtual reality email and advertisement service system and method using real-time image synthesis"

The technical problem to be achieved by the present invention is to provide a method and a system capable of generating a matching image simply and relatively accurately using a terminal carried by a user.

In particular, by using a driving device capable of rotating the terminal, it is possible to relatively accurately know the direction of an image to be matched, and thereby provide a method and system capable of relatively fast and accurate image registration when matching images. .

In addition, the terminal performs rotation to photograph images to be matched, that is, partial images, but not only images obtained through one rotation are used to generate a matched image, but is partially divided for each rotation through a plurality of rotations. It is to provide a method and system for acquiring images and generating a matched image by using partial images obtained from different rotations together.

It is also to provide an advertising method suitable for the characteristics of a 360 image or a matched image, and a system therefor.

The advertisement providing method according to an embodiment of the present invention for achieving the above technical problem may include determining, by a system, a display area identified by a user among 360 images, and a predetermined advertisement content may be disposed in the display area determined by the system. Adding to the advertisement area, and placing the predetermined advertisement content in the current advertisement area by the system.

The 360 image corresponds to a certain angle and is an image in which n partial images facing different directions are matched, and the display area identified by the user may be determined in units of the partial image.

The advertisement providing method using the 360 image further includes determining a placement location in which the advertisement content is to be placed in the advertisement area, and disposing the advertisement content in the determined placement location. In the step of determining the placement location where the advertisement content is to be arranged, the system recognizes the object location where the object corresponding to the predetermined advertisement content is displayed, and determines the placement location corresponding to the recognized object location. You can.

The advertisement providing method using the 360 image further includes determining a placement location in which the advertisement content is to be placed in the advertisement area, and disposing the advertisement content in the determined placement location. In the step of determining the placement location where the advertisement content is to be placed, it may be determined that the non-critical region in which the advertisement content can be displayed-the non-critical region is determined based on image complexity-as the placement location.

The 360 image corresponds to a certain angle and is an image in which n partial images facing different directions are matched, and determining a placement position in which the advertisement content is to be placed in the advertisement area includes k of the partial images. Before photographing a partial image (k is an integer from 2 to n), it may be characterized in that it starts to determine the arrangement position included in the (k-1) partial image.

The step of placing predetermined advertisement content in the current advertisement area by the system may include disposing the advertisement content in an advertisement area that is not currently included in the user's field of view among the advertisement areas, and is not included in the view of the user After placing the advertisement content in the advertisement area, it may include the step of outputting a notification signal indicating that the advertisement content is placed.

Another example of a method for providing advertisement for implementing the technical idea of the present invention is a step in which a system starts displaying a 360 image on a user's terminal, while the system displays the 360 image while displaying predetermined advertisement content in the 360 And disposing the advertisement content at a predetermined placement location included in the 360 image, placing the advertisement content in an area not included in the current field of view of the user. It is characterized by being arranged.

Another example of an advertisement providing method for implementing the technical idea of the present invention is a method in which the system displays a matched image in which n partial images matched to a certain angle and facing different directions, and the system displays the advertisement And determining a placement position where the content is to be placed, wherein determining the placement position is performed before taking a k-th partial image among the partial images (k is an integer from 2 to n). It may be characterized by starting to determine the placement position included in the partial image.

The above method can be implemented by a computer program recorded on a computer-readable recording medium.

An example of a system for implementing the technical idea of the present invention includes a control unit including a processor and a storage medium in which a program executed by the control unit is recorded, and the program determines a display area identified by a user among 360 images , The determined display area is added as an advertising area in which predetermined advertising content can be placed, and predetermined advertising content is disposed in the current advertising area.

In addition, the system according to another example includes a control unit including a processor and a storage medium on which a program executed by the control unit is recorded, wherein the program starts to display a 360 image on the user's terminal by the system, and the 360 image It is characterized in that while displaying a predetermined advertisement content is placed in a predetermined placement position included in the 360 image, the placement position is an area not included in the current field of view of the user.

In addition, the system according to another example includes a control unit including a processor and a storage medium in which a program executed by the control unit is recorded, wherein the program corresponds to a certain angle and matches n partial images facing different directions. Before displaying the k-th partial image among the partial images (k is an integer from 2 to n) to display the matched image, determine the placement location where the advertisement content will be placed, and determine the placement location (k-1) It is characterized in that it starts to determine the arrangement position included in the partial image.

According to the technical idea of the present invention, by allowing a user to generate a simple and relatively accurate matched image using a terminal carried by a user, a matched image can be easily generated by anyone at a low cost (for example, a 360 image that can be utilized for VR). There is an effect that can be used.

In addition, by using a driving device capable of rotating the terminal, the direction of the image to be matched can be known relatively accurately, and through this, a relatively fast and accurate image matching is possible when matching the image. Since the control of the direction (rotation angle) at the time of photographing of the image can be accurately performed, the terminal does not use only the images to be matched, that is, images obtained through one rotation to shoot partial images, to generate the matched image, There is an effect of acquiring partial images for each rotation through a plurality of rotations, and generating a matched image by using the partial images obtained from different rotations together.

Through this, for example, when there is an image difference between a partial image photographing a specific direction in the first rotation (eg, a front direction) and a partial image photographing the specific direction in the second rotation, a desired partial image is photographed in another rotation. It is possible to obtain an effect desired by a user by matching with partial images. For example, in the partial image corresponding to a specific direction in the first rotation, there are features that the user does not want (eg, a specific object (eg, a person, etc.) is captured), and in the second rotation, the characteristic does not exist in the specific direction. If not (e.g., a specific object (e.g., a person, etc.) is not photographed), the overall registration is achieved by combining images excluding the partial image in the specific direction in the first rotation and the partial image in the specific direction in the second rotation This has the effect of preventing unwanted features from appearing in the image.

In addition, by allowing the advertisement to be attached to the 360 image in real time according to the user's action in the 360 image, it is possible for the user to have no inconvenience in checking the original image of the 360 image.

In addition, by allowing advertisements to be placed in real time in an area independent of the current scene, the user's usability can be increased.

In addition, the location where the advertisement content is to be placed can be previewed before the individual partial image is captured and before the 360 image is generated, so that the real-time 360 image generation and advertisement utilizing the same can be effectively performed.

BRIEF DESCRIPTION OF THE DRAWINGS In order to better understand the drawings cited in the detailed description of the present invention, a brief description of each drawing is provided.
1 is a diagram illustrating devices for implementing a method for generating a matched image according to an embodiment of the present invention.
2 shows a schematic configuration of a terminal according to an embodiment of the present invention.
3 is a diagram for describing a matched image and a partial image according to an embodiment of the present invention.
4 is a flowchart illustrating a method for generating a matched image according to an embodiment of the present invention.
5 is a view for explaining a concept of acquiring partial images by performing a plurality of rotations using a matching image generating method according to an embodiment of the present invention.
6 is a view for explaining a concept of generating a matched image by matching partial images obtained from each of a plurality of rotations through a matched image generating method according to an embodiment of the present invention.
7 to 9 are views for explaining a concept of generating a matched image by matching partial images obtained from each of a plurality of rotations through a matched image generating method according to another embodiment of the present invention.
10 is a view for explaining an advertising technique using a display area according to an embodiment of the present invention.
11 is a view for explaining a placement position of advertisement content according to an embodiment of the present invention.
12 is a diagram for explaining a concept in which advertisement content is disposed in a 360 image in real time outside the user's view according to an embodiment of the present invention.

In order to fully understand the present invention, the operational advantages of the present invention, and the objects achieved by the practice of the present invention, reference should be made to the accompanying drawings and the contents described in the accompanying drawings, which illustrate preferred embodiments of the present invention.

In addition, in the present specification, when one component 'transmits' data to another component, the component may directly transmit the data to the other component, or through at least one other component It means that the data may be transmitted to the other components. Conversely, when one component 'directly transmits' data to another component, it means that the data is transmitted from the component to the other component without passing through the other component.

Hereinafter, the present invention will be described in detail by explaining preferred embodiments of the present invention with reference to the accompanying drawings. The same reference numerals in each drawing denote the same members.

1 is a diagram illustrating devices for implementing a method for generating a matched image according to an embodiment of the present invention.

Referring to FIG. 1, a terminal 100 may be provided to implement a method of generating a matched image according to the technical idea of the present invention.

The terminal 100 may be meant to include all types of data processing devices equipped with an image sensor capable of being carried by a user and acquiring an image. According to an example, the terminal 100 may be a user's mobile phone, but is not limited thereto. In addition, a part of the technical idea of the present invention may be implemented by a device that the user does not carry and may be included in the scope of the present invention.

The terminal 100 may take images (partial images) while performing rotation. The terminal 100 may or may not rotate 360 degrees around the terminal 100. In any case, the terminal 100 may rotate as necessary so that the registration image to be generated can cover all the angles (eg, 360 degrees, 270 degrees, 180 degrees, etc.) to be covered. Of course, the degree of rotation may vary depending on the size of the viewing angle of the image sensor provided in the terminal 100.

The terminal 100 may be attached, fastened, or coupled to a predetermined driving device 200 that rotates the terminal 100. In addition, movement of the terminal 100 may be performed through the driving device 200.

The terminal 100 may control the driving device 200. According to an example, the terminal 100 may perform a predetermined short-range wireless communication (eg, Bluetooth, NFC, Zigbee communication, WiFi, etc.) with the driving device 200. The terminal 100 may transmit a predetermined command to the driving device 200, and the driving device 200 may drive in response to the transmitted command. Also, according to an embodiment, the driving device 200 may transmit a driving result of the driving performed in response to the command to the terminal 100.

The driving device 200 may be a device that performs at least a rotational motion. A rotator for such rotational movement may be provided in the driving device 200. In addition, the driving device 200 may be provided with a predetermined fastening part (not shown) for fastening the terminal 100.

According to an example, the driving device 200 may be a device capable of performing rotation in a predetermined reference angle unit, and rotates as much as an angle desired by the command in response to a command received from the terminal 100. Can be. Through this method, the rotation angle of the driving device 200 and / or the terminal 100 may be controlled.

According to another embodiment, the driving device 200 may perform rotation in response to a command received from the terminal 100. In addition, when the terminal 100 senses the degree of rotation of the driving device 200 (ie, the terminal 100) and senses that the required angle is rotated, a command to stop rotation of the driving device 200 may be transmitted. have. In any case, the terminal 100 can control its rotation angle relatively accurately through control of the rotation angle of the driving device 200.

In the present specification, an example in which the driving device 200 simply has a function of rotating the terminal 100 is described, but if necessary, the driving device 200 moves and tilts the terminal 100. ), Or may be implemented to perform functions such as changing the position of the terminal 100 vertically.

Meanwhile, a predetermined viewing angle magnification device (eg, a fisheye lens, etc. 300) for enlarging a viewing angle in a specific direction (eg, vertical direction) may be attached to the terminal 100. In this case, a viewing angle of up to 180 degrees may be secured up and down by the viewing angle enlargement device. In this case, when the terminal 100 rotates to secure a viewing angle of 360 degrees to the left and right, a 360 degree image (spear Of course, it is possible to create an image).

After all, according to the technical idea of the present invention, the terminal 100 controls the driving device 200 to rotate the image relatively accurately by a desired angle (ie, rotates in a desired direction), and then shoots an image again, and then again by a desired angle Rotation can be performed to take an image. When the control of the rotation angle is relatively accurately performed by the terminal 100, the terminal 100 may accurately know the rotation angle (viewing angle) of an adjacent image, and in this case, the accuracy and speed of image registration (stitching) There is an effect that can be improved.

On the other hand, according to another embodiment, the terminal 100 simply performs a rotation to cover a desired angle once, and matches the plurality of partial images obtained during the rotation to generate a matched image, but not a plurality of rotations. While performing, at least one partial image may be obtained for each rotation, and partial images obtained in different rotations may be matched to generate one matching image.

The technical idea may provide a technical idea of generating a matched image by replacing a partial image of a specific direction acquired in a specific time period with a partial image of the specific direction acquired in another time period. In addition, this technical idea can be further increased when the rotation of the terminal 100 is accurately controlled through the driving device 200 as described above.

In addition, the terminal 100 may provide advertisement content as will be described later. For example, the terminal 100 may display a 360 image and perform an advertisement method according to the technical idea of the present invention. Of course, according to another embodiment, the predetermined service server 400 may perform the advertisement method according to the embodiment of the present invention. Alternatively, the terminal 100 and the server 400 may be organically combined to perform an advertising method according to the technical idea of the present invention.

The advertisement method according to the technical concept of the present invention may provide a concept specialized in an image in which an entire image (eg, a 360 image or a 270 degree image) cannot be displayed in one field of view. Hereinafter, the concept of the advertisement method according to the technical idea of the present invention will be described to be applied to a 360 image, but the average of the technical field of the present invention can be widely used for an image in which the entire image cannot be displayed in one view of the user. Experts will be able to reason easily.

The advertisement method has a feature of setting the starting area as an advertising area, that is, an area in which advertising content can be arranged. That is, it is possible to ensure that the advertisement content is not placed in an area in which the user has never checked the original image, so that at least the user can see the original image once for the entire image area.

In addition, due to this feature, the advertisement method may have a feature in which advertisement content is disposed in a part of the 360 image in real time while the user is viewing the 360 image. Moreover, the advertisement content may be arranged outside the user's field of view.

The configuration of the terminal 100 for implementing the technical idea will be described with reference to FIG. 2.

2 shows a schematic configuration of a terminal according to an embodiment of the present invention.

Referring to FIG. 2, the terminal 100 includes at least a control unit 110, an audio / video (A / V) input module 120, and a storage medium 160. The terminal 100 may further include a communication module 130, an output module 140, and an input module 150. The terminal 100 may communicate with a predetermined service server 400. In this case, of course, the communication module 130 may be provided with a wireless Internet device or the like for performing communication with the service server 400.

In addition, the components shown in FIG. 2 are not essential, so a mobile terminal having more or fewer components may be implemented.

In the present specification, the module means hardware for performing the technical idea of the present invention (eg, the control unit 110 and / or storage medium 160) and software for driving the hardware (eg, the technology of the present invention. It may mean a functional and structural combination of the above program) for realizing the idea. For example, each of the components may mean a logical unit of a predetermined code and a hardware resource for performing the predetermined code, and must mean a physically connected code, or one type or a specific number of hardware. It does not mean that can be easily inferred to the average expert in the technical field of the present invention. Accordingly, each of the above configurations refers to a combination of hardware and software performing functions defined herein, and does not mean a specific physical configuration.

The A / V (Audio / Video) input module 120 includes at least an image sensor 121. The image sensor 121 may mean a device capable of capturing / acquiring an image, and according to an implementation example, the A / V (Audio / Video) input module 120 may include a plurality of image sensors 121 It might be.

In addition, according to the implementation example, the A / V (Audio / Video) input module 120 may further include another configuration (eg, a microphone) for inputting an audio signal or a video signal.

The communication module 130 includes a short-range communication module 131. In addition, as described above, if necessary, a wireless communication device (not shown) may be further provided to perform communication with an external wireless communication system.

The terminal 100 may communicate with the driving device 200 through the short-range communication module 131 included in the communication module 130. The short-range communication protocol performed by the short-range communication module 131 is, for example,

Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, WiFi, and the like can be used.

The output module 140 may be configured to generate output related to vision, hearing, or tactile sense. For example, a display device, a sound output device, an alarm device, and a haptic module may be included.

The terminal 100 may display a matched image or a 360 image through the output module 140. In addition, the control unit 110 may determine whether the area displayed by the output module 140 is the matched image or the 360 image. As described above, the area identified by the user (the area corresponding to the area displayed through the output module 140 is referred to as a start area.

The display area may mean an area actually displayed by the output module 140 regardless of a partial image constituting a matched image or a 360 image (that is, only a part of the partial image may be a display area), Depending on the embodiment, the starting area may be determined for each partial image.

In addition, advertisement content may be arranged at a predetermined placement position of the matched image or the 360 image using the grasping result of the display area.

Of course, the placement of the pandas and advertisement contents in such a display area may be performed by the service server 400 to which the terminal 100 for reproducing a 360 image is connected.

The input module 150 is implemented to receive input signals desired by the user through the terminal 100. For example, it may be composed of a key pad, a dome switch, a touch pad (static pressure / electrostatic), a jog wheel, a jog switch, or the like.

The storage medium 160 may be a device capable of permanently or temporarily storing data generated by the terminal 100 or input from the outside in order to implement the technical idea of the present invention.

According to an embodiment, the storage medium 160 may be divided into a plurality of different physical devices, and according to an embodiment, a part of the storage medium 160 may exist inside the control unit 110. The storage medium 160 may be implemented as a hard disk, a solid state disk (SSD), an optical disk, a random access memory (RAM), and / or various other types of storage media, depending on implementation. The terminal 100 may be implemented detachably.

According to an example, a program for implementing the technical idea of the present invention may be stored in the storage medium 160. The program may be driven (executed) by the control unit 110, and functions defined in the present invention may be performed by executing the program. Hereinafter, in this specification, it may mean that a function defined in this specification is performed by the program, unless otherwise specified.

The control unit 110 includes a processor and can control the overall operation of the terminal 100. The control unit 110 may include a processor, a micro process, etc., and may implement the technical idea of the present invention by driving a program stored in the storage medium 160 as described above.

In addition, if necessary, the terminal 100 may be provided with a predetermined sensing module (not shown), the opening and closing state of the terminal 100 through the sensing module, the location of the terminal 100, the presence or absence of user contact, It is possible to sense the orientation of the terminal and acceleration / deceleration of the terminal. In addition, as described above, by sensing the rotation angle of the terminal 100, the terminal 100 may be controlled based on the sensed information.

According to an example, the terminal 100 may acquire partial images while performing rotation by controlling the driving device 200. Matching of the obtained partial images may be performed, and matching of the partial images may be performed in the terminal 100. However, if necessary, matching of partial images may be performed by the service server 400 that communicates with the terminal 100, in which case the terminal 100 transfers the obtained partial images to the service server 400. Can transmit. In addition, if necessary, a matched matching image may be received from the service server 400 and displayed on the terminal 100.

3 is a diagram for describing a matched image and a partial image according to an embodiment of the present invention.

Hereinafter, in the present specification, unless explicitly described otherwise, the matched image matched by the terminal 100 or the service server 400 may mean a 360-degree image (spear image). In addition, the terminal 100 will be described as an example of taking six different partial images while rotating 360 degrees by rotating by 60 degrees. In addition, the function of matching the partial images is also exemplarily described when the terminal 100 performs, but the matching of the partial images can be performed by the service server 400 as described above.

Referring first to FIG. 3A, the terminal 100 may photograph n (eg, 6) partial images (eg, 10-1 to 15-1) to generate a matched image. To this end, the terminal 100 may control the driving device 200 to rotate to n directions (eg, 10 to 15), and then take a partial image in each direction.

For example, the terminal 100 may photograph the first partial image 10-1 in the first direction 10 and control the driving apparatus 200 to face the second direction. In addition, the second partial image 11-1 may be photographed in the second direction 11. In this way, the terminal 100 may sequentially acquire six partial images. In this case, the image sensor 121 of the terminal 100 may preferably have a viewing angle greater than at least 60 degrees left and right. Since the terminal 100 can sequentially acquire partial images (eg, 10-1 to 15-1) through rotation and photographing, it is possible to know what partial images are adjacent to each other. Moreover, since the direction of each partial image is accurately known, it is possible to easily determine a region overlapping between the partial images, and there is an effect that image registration can be achieved in a relatively fast time.

In order for the terminal 100 to perform rotation, the terminal 100 may transmit a command to rotate a predetermined angle (eg, 60 degrees) to the driving device 200. Then, in response to a command, when receiving a signal from the driving device 200 that the rotation has been performed by the angle, the terminal 100 may take a partial image.

According to another embodiment, the terminal 100 may output a command to rotate the driving device 200 to the driving device 200. When the driving device 200 rotates in response to this, the terminal 100 may sense the rotation angle. In addition, when the rotation angle corresponds to a desired angle, the terminal 100 may output a command for stopping rotation to the driving device 200. Then, the driving device 200 may stop rotating, and then the terminal 100 may take a partial image.

In any manner, the terminal 100 may rotate a desired angle to take a partial image at the performed angle (direction).

On the other hand, the technical idea of the present invention may not necessarily mean that the terminal 100 should be rotated 360 degrees to the left or right, and may also rotate only the angle necessary to generate a matched image covering a predetermined angle. An example of this may be as shown in FIG. 3B.

As illustrated in FIG. 3B, the terminal 100 may generate a registration image covering 180 degrees, and for this purpose, sequentially to face n directions (eg, 12 directions including 21) divided by 180 degrees. It is possible to photograph n partial images (eg, 12 partial images including 21-1) in the n direction while rotating. In addition, a matched image covering 180 degrees may be generated by matching the partial images.

The method of generating a matched image of the present invention can be briefly summarized as in FIG. 4.

4 is a flowchart illustrating a method for generating a matched image according to an embodiment of the present invention.

Referring to FIG. 4, the terminal 100 may acquire a partial image in a predetermined direction (S10). And it is possible to perform rotation by a predetermined angle by controlling the driving device 200 (S20). In addition, a corresponding partial image may be obtained in a state in which rotation is performed (S30). In addition, it may be determined whether n partial images corresponding to the desired n directions are obtained (S40). If it is determined that all n partial images have not been acquired (S40), the terminal 100 may obtain a partial image after performing rotation by controlling the driving device again (S20, S30).

If it is determined that all n partial images have been acquired (S40), the terminal 100 may perform a matching process of matching the acquired partial images (S50). The matching process may be a process in which the terminal 100 directly performs image registration, or the terminal 100 may transmit partial images to the service server 400 and allow the service server 400 to match partial images. As mentioned above, it can mean a process.

On the other hand, the technical idea of the present invention, as described above, the terminal 100 by performing a plurality of rotation, it is possible to use the partial images obtained through different rotations to generate a single registration image.

An example of this will be described with reference to FIGS. 5 to 6.

5 is a view for explaining a concept of acquiring partial images by performing a plurality of rotations using a matching image generating method according to an embodiment of the present invention. In addition, FIG. 6 is a diagram for explaining a concept of generating a matched image by matching partial images obtained from each of a plurality of rotations through a matched image generating method according to an embodiment of the present invention.

First, referring to FIG. 5A, the terminal 100 rotates to face n (eg, 6) directions (eg, 10 to 15), respectively, and then partial images in each direction (eg, 10 to 15). Fields (eg, 10-1 to 15-1) can be photographed. This rotation and shooting process is defined as a basic rotation.

Then, the terminal 100 does not match the partial images obtained through the basic rotation (eg, 10-1 to 15-1) to generate a matching image, but may perform additional rotation. In addition, at least one partial image may be acquired through additional rotation. The additional rotation may be one or more times, but for convenience of description, it is assumed that the additional rotation is once.

In addition, the partial image obtained in the basic rotation will be defined as the basic rotation partial image, and the partial image obtained through the additional rotation will be defined as the additional rotation partial image.

Then, the technical idea of the present invention may generate a matched image by matching all or part of the partial rotational images and the additional rotational partial images. Through this, a concept is proposed in which images captured in different time periods (for example, a time period corresponding to a basic rotation and a time period corresponding to an additional rotation) are matched to generate a single matched image.

This concept is intended to generate a matched image (eg, a 360-degree image) by capturing a static subject (eg, landscape, building, real estate, etc.), that is, a subject that is not significantly affected by time. When the corresponding basic rotation partial image has a feature that the user does not want, it is possible to provide an effect of replacing the basic rotation partial image with an additional rotation partial image without the feature. In this case, the additional rotating partial image to be replaced may be a partial image (ie, a partial image corresponding to the same direction) corresponding to the basic rotating partial image to be replaced.

For example, an unwanted feature (eg, a specific object is photographed) may exist in the third partial image 12-1 obtained in the third direction 12 in FIG. 5A.

Then, if the terminal 100 performs additional rotation as shown in FIG. 5B, n additional rotation partial images (eg, 10-2) for n directions (eg, 10 to 15) identical to the basic rotation. 15-2), the additional rotation third partial image 12-2 is replaced by the basic rotation third partial image 12-1 in which the feature does not exist (for example, a specific object is not photographed). Can be used for matching.

That is, among the basic rotation partial images 10-1 to 15-1, the basic rotation matching target image that is the object of registration is the basic rotation partial images (eg, 10-) except for the third partial image 12-1. 1, 11-1, 13-1, 14-1, 15-1). In addition, the additional rotational partial image that is the object of registration may be an additional rotational partial image 12-2 corresponding to the basic rotational third partial image 12-1, which is the basic rotational partial image to be replaced.

Then, the result of matching the basic rotation matching target image (for example, 10-1, 11-1, 13-1, 14-1, 15-1) and the additional rotation matching target image (for example, 12-2) is shown in FIG. 6. A registration image as shown can be generated.

If the feature still exists in the third partial image (eg, 12-2) of the additional rotation obtained through the additional rotation (eg, the second rotation), the terminal 100 again rotates the additional rotation (eg, the third rotation). You can do In any case, an image of the third part of the additional rotation, in which the above feature does not exist, can be obtained regardless of the number of additional rotations and can be set as a matching object.

For example, it may be assumed that the feature is a feature in which a specific object (eg, a person) is photographed as described above, and that the terminal 100 excludes all the basic rotational partial images in which the feature exists. . Then, among the basic rotational partial images, all partial images that are targets for registration may be images in which the feature does not exist. In addition, the additional rotated partial images corresponding to each of the excluded basic rotated partial images may be partial images without the feature. In this way, if the partial images in which the feature does not exist among the basic rotational partial images, that is, the additional rotational matching target image among the basic rotational matching target image and the additional rotational partial images, the matched registration image is the characteristic in all directions This does not exist.

For example, assuming that the terminal 100 photographs a specific space and the feature is that a person is photographed, if a person is photographed in a partial image (eg, 12-1) in a specific direction through a basic rotation, the The basic rotational partial image (eg, 12) in which a person is photographed by photographing the additional rotational partial image (eg, 12-2) in which a person is not photographed for the specific direction is photographed -1). And if the replaced basic rotation partial images (for example, 10-1. 11-1, 12-2, 13-1, 14-1, 15-1) are matched, a matching image covering the space in all directions is generated. As a result, it may have a characteristic that a person is not photographed in any direction.

The partial image having a characteristic among the basic rotational partial images may be excluded from registration. For example, as described above, it may be the characteristic that the specific object is photographed, or, conversely, that the specific object is not photographed. Various features may be defined according to an implementation example, and according to the definition of these features, there is an effect of generating a matched image in which the feature does not exist in all directions covered by the matched image.

In addition, the matched image may have an effect of generating new space-time by matching partial images of different time zones.

On the other hand, while performing the additional rotation, as shown in FIG. 5B, the terminal 100 also has n additional rotational partial images (eg, 10-) corresponding to each of the n directions as in the basic rotation even when the additional rotation is performed. All 2 to 15-2) may be photographed.

In addition, as described above, the basic rotation matching image (for example, 10-1. 1, 13-1, 14-1, 15-1) and the n additional rotated partial images (eg, 10-2 to 15-2) corresponding to the excluded partial image (eg, 12-1) The additional rotation matching image, which is an image (eg, 12-2), may be registered.

That is, even in the additional rotation, the terminal 100 may photograph all n images corresponding to each of the predefined n directions.

In this case, the terminal 100 may include the basic rotation partial images (eg, 10-1 to 15-1) obtained in the basic rotation and additional rotation partial images (eg, 10) corresponding to each of the basic rotation partial images. -2 to 15-2) may be compared to determine a partial image to be excluded.

For example, six basic rotation partial images may be obtained for six different directions in the basic rotation, and six additional rotation partial images may be obtained in six different directions in the additional rotation. In addition, the terminal 100 may compare each of the corresponding partial images (eg, 10-1 and 10-2, 11-1 and 11-2). In addition, when the comparison result satisfies a predetermined criterion, the image of the basic rotation part in the corresponding direction may be replaced with an additional rotation part image.

For example, the terminal may obtain a differential image of partial images corresponding to each other, and if the difference between the partial images is greater than or equal to a certain level by using the differential image, the terminal may determine that the basic rotational partial image in the corresponding direction is excluded have.

In the end, the embodiment for selecting the partial image to be excluded may be based on whether the partial image has a predetermined characteristic as described above, or may be determined by comparison of corresponding partial images, and the user directly rotates the basic rotation portion. It is also possible to select at least one of the images or the additional rotating partial images as a partial image to be replaced (replaced). Various embodiments may be possible.

On the other hand, according to another embodiment, the terminal 100 may not necessarily need to acquire additional rotational partial images in all of the n directions through additional rotation. That is, depending on the embodiment, only the additional rotational partial image corresponding to the excluded partial image among the basic rotational partial images may be obtained through additional rotation.

In determining the partial image to be excluded, the terminal 100 analyzes each of the basic rotational partial images, and as described above, which partial image has a predetermined characteristic (satisfied) (which direction out of n directions)? It can be achieved by judging.

Or, the user may be provided with the basic rotational partial images, and in response, at least one of the basic rotational partial images may be selected. Then, the selected basic rotational partial image may be selected as the excluded partial image.

When the partial image to be excluded is determined as described above (ie, a direction corresponding to the partial image to be excluded among n directions is determined), the terminal 100 causes the terminal 100 to rotate in the specific direction, and then the direction. It is possible to acquire an image of the additional rotation part in. In addition, the obtained partial image of the additional rotation may be the image of the additional rotation matching object used for generating the registration image.

An example of the concept of selectively acquiring an additional rotational partial image in only some of the n directions will be described with reference to FIGS. 7 to 9.

7 to 9 are diagrams for explaining a concept of generating a matched image by matching partial images obtained from each of a plurality of rotations through a matched image generating method according to another embodiment of the present invention.

First, referring to FIG. 7, the terminal 100 may provide all (eg, 10-1 to 15-1) or part of the basic rotation partial images to the user. The user may select at least one of the provided basic rotational partial images as an exclusion partial image (eg, 12-1, 14-1). Alternatively, the terminal 100 may analyze each of the basic rotation partial images and select a partial image (eg, 12-1, 14-1) having a predetermined characteristic as the excluded partial image.

Then, as illustrated in FIG. 8, the terminal 100 may selectively rotate in each direction (eg, 12, 14) corresponding to the excluded partial image. In addition, the additional rotated partial images 12-3 and 14-3 may be acquired only in the rotated directions (eg, 12 and 14). That is, in this case, in the directions 10, 11, 13, and 15, an additional rotational part image may not be captured.

Then, among the basic rotation partial images, the basic rotation matching target images 10-1, 11-1, 13-1, and 15-1, excluding the excluded partial images 12-1 and 14-1, and the excluded partial images The additional rotation partial images 12-3 and 14-3 corresponding to (12-1 and 14-1) are matched to generate a matched image.

Meanwhile, the technical idea as described above is also applied to FIG. 9, in which the angle and n covered by the registration image are different from each other, so that partial images obtained from different rotations may be included in one registration image.

That is, in FIG. 9, the terminal 100 may acquire 12 partial images in 12 directions through the basic rotation. Then, through the additional rotation, 12 additional rotational partial images are obtained for the same 12 directions, or an additional rotational partial image corresponding to an excluded partial image among the basic rotational partial images (eg, 24-2, 29-2) You can also acquire bays.

And the basic rotation matching target image (21-1, 22-1, 23-1, 25-1, 26-1, 27-1, 28-1, 30-1, 31-1, 32-1) and additional rotation The registration target images 24-2 and 29-2 may be matched to generate a registration image.

As a result, according to the method for generating a matched image according to the technical idea of the present invention, partial images of different time zones are included as components of one matched image, thereby generating a new space-time. That is, a part of the registration image represents a space in the first time zone (time section), and the rest represents a space in a time zone different from the first time, thereby bringing various effects.

In addition, the method for generating a matched image having the effect of generating a new space-time as described above does not necessarily require the driving device 200. That is, an average expert in the technical field of the present invention can easily infer that the above-described technical idea can be achieved even when a person directly rotates the terminal 100 through a predetermined guide or interface. However, it is a matter of course that the performance (quality and speed) of image matching may be much higher by controlling the rotation angle accurately through the driving device 200.

Meanwhile, a technical idea of providing an advertisement to the user while a user consumes a matched image or a 360 image matched in the manner described above will be described with reference to FIGS. 10 to 12. Of course, the advertising method to be described later is not necessarily applied only to the matched image generated in the manner described above, and the average expert in the technical field of the present invention can easily infer that it can be applied to the matched image or the 360 image generated in other ways You will be able to.

In addition, as described above, the advertisement method according to the technical idea of the present invention may be performed in the terminal 100 or may be performed in the service server 400. That is, the determination of the starting area as described later, the determination of the placement location, and the placement of advertisement content may be performed in the terminal 100 or may be performed in the service server 400. Alternatively, it may be performed by a completely different system independent of the generation of the matched image or the 360 image.

Therefore, in the following description, an apparatus capable of performing an advertising method according to the technical idea of the present invention is simply referred to as a system, and the system may be the terminal 100 and / or the service server 400, The average expert in the technical field of the present invention can easily infer that it may be a completely separate device.

Hereinafter, for convenience of description, a case in which the terminal 100 performs the advertisement method will be described.

10 is a view for explaining an advertising technique using a display area according to an embodiment of the present invention.

Referring to FIG. 10, the system may receive a request to play a 360 image and play (display) the 360 image. The system (eg, the output module 140 of the terminal 100) may display only a portion of a 360 image corresponding to a preset viewing angle.

According to an example, the displayed area may be treated as an area identified by the user. As described above, in this specification, an area visually confirmed by the user through the system will be defined as a start area.

The display area may mean an area currently displayed on a display device connected to the system, that is, a view plane itself. However, the display area may be determined in units of partial images constituting the 360 image.

In this case, it may mean that the entire partial image must be displayed at least once, so that the corresponding partial image can be determined as the start area. Also, even if a specific area displayed by the system is not displayed on the entire partial image including the specific area, it may mean that the specific area is not included as a starting area.

As described above, when determining the starting area as a partial image, that is, a matching target image unit before registration, the process can be performed in a partial image unit, thereby making it possible to perform computation or process efficiency.

Of course, the system can change the field of view while rotating the 360 image according to a user's request, and of course, can determine the starting area in real time.

For example, FIG. 10A shows a case in which the first partial image 10-1 is determined as a starting area, and in FIG. 10B, the first partial image 10-1 and the second partial image 11-1 are starting areas. It may indicate a case judged as.

As described above, according to the technical idea of the present invention, the starting area can be determined in real time, and accordingly, an advertisement can be placed only in the starting area. That is, among the respective regions included in the 360 image, it may mean that the advertisement is placed only in the portion identified by the user at least once, and thereby, the advertisement content does not interfere with the user's space confirmation. There is.

In addition, it may mean that the advertisement content may be arranged in a 360 image that is already being played in real time while the user is playing the 360 image by determining the start area in real time.

The system may add an area determined as a start area in the 360 image during playback as an advertisement area. That is, it can be treated as an area where advertising content can be placed. To this end, the system (eg, the control unit 110 of the terminal 100) may maintain / manage information on the advertisement area in real time. Of course, the list of the display area is the same as the list of the advertisement area, but conceptually, the display area is expressed as an advertisement area to clarify that the display area is an advertisement area.

Then, the system may place predetermined advertisement content at a predetermined position in the advertisement area.

Although the system may place advertisement content at any location included in the advertisement area, a location that satisfies a predetermined criterion according to an embodiment of the present invention, that is, a placement location is searched within the advertisement area and the placement location is searched The advertisement content may be arranged in the.

An example of this will be described with reference to FIG. 11.

11 is a view for explaining a placement position of advertisement content according to an embodiment of the present invention.

Referring to FIG. 11, a first partial image 10-1 may be included in an advertisement area (that is, a display area), and the first partial image 10-1 may be an object (eg, as shown in FIG. 11). , 40, 41, 42).

According to an example, the placement location corresponds to advertisement content (eg, a TV image to be advertised) to be placed, and may correspond to a location of an object (TV displayed in the 360 image, 41) in the advertisement area. . That is, the placement position may be the same as the position of the object or a position that can be adjusted little by little according to an embodiment, such as a wider area (or a narrower area) than the object.

In this case, in the end, the system has an effect of replacing an object (eg, 41) existing in the 360 image with advertisement content (eg, a TV image targeted for advertisement).

To this end, the system (eg, the controller 110) may perform a process of recognizing an object corresponding to a predetermined advertisement content in the 360 image. And object recognition may be possible through various embodiments. According to an example, various object recognition algorithms based on feature points have been known. Also, in recent years, object recognition algorithms using deep learning (eg, TensorFlow, etc.) have been widely known. The average expert in the technical field of the present invention can easily infer that various object recognition methods can be used to implement the technical idea of the present invention.

In addition, the system (eg, the controller 110) may maintain / manage information (eg, images, types of objects, etc.) for advertisement contents in advance and perform object recognition using the information.

According to another embodiment, the arrangement position is present in the advertisement area, and may be a non-critical area determined by a standard according to the technical idea of the present invention.

The non-critical area may refer to an area where important information does not exist even though advertisement content is disposed to cover the corresponding area according to an embodiment of the present invention.

As such, there may be a variety of ways to judge non-critical areas.

According to an example, the non-critical area may be an area determined by the system (eg, the control unit 110) and an image complexity of a predetermined value or less.

In addition, it is known that image complexity can be measured in various ways, such as a difference between a pixel value or a luminance value of a pixel, and a differential between a specific pixel and surrounding pixels. Also, the relatively low image complexity may mean that there is not much visual change in the corresponding area, that is, the same or similar information is continuously present. For example, continuous areas in which the difference in pixel information is not large may be relatively low in image complexity, and these areas may be, for example, the sky, the sea, the surface of the door 40, the top surface of the desk 42, and the like.

Therefore, the system (for example, the controller 110) uses the image complexity as described above to search for an area where the image complexity is relatively low (based on a predetermined reference value), and an area having a predetermined size or more among the searched areas as the placement position. I can judge.

To this end, the system (eg, the controller 110) calculates image complexity in macro convex units included in the advertisement area, and finds an area in which blocks lower than a reference value are continuously arranged using the concatenated image complexity. Of course, the non-critical region can be searched using image complexity in various ways.

According to an example, the system (eg, the control unit 110) determines a placement position of an upper surface of the table 42 included in the first partial image 10-1, which is an advertisement area, as a placement position of specific advertisement content. You can. In addition, the specific advertisement content may be disposed on the upper surface of the table 42.

On the other hand, the system (eg, the controller 110) may further provide a technical idea for effectively performing the above-described methods of determining the placement position (location or non-critical area of the corresponding object).

According to an example, the system (eg, the control unit 110) excludes a region 10- except for a stitching region (ie, an area overlapping with an adjacent image, for example, 10-11, 10-12) among advertisement regions. 10) The placement position can only be determined within. Excluding the stitching area (eg, 10-11, 10-12) is because it is not desirable to replace or overwrite the stitching area (eg, 10-11, 10-12) containing information important for stitching the image. You can. In addition, when the stitching area (eg, 10-11, 10-12) is excluded, it can be very effective because it is possible to search for the arrangement position in advance.

That is, if the stitching area (eg, 10-11, 10-12) is excluded, as described above, after the first partial image 10-1 is photographed, the second partial image 11-1 may be photographed. In advance, even if the first partial image 10-1 is photographed before the second partial image is photographed, or even before the first partial image 10-1 is stitched with another image, the placement location of the advertisement content can be previewed in advance. It has the effect of being able to navigate.

On the other hand, the technical idea of the present invention can be arranged in real time advertising content while the user is already viewing the 360 image as described above. In this case, if an advertisement is artificially placed in an area that the user is already viewing, that is, the area being displayed, it may be very unnatural or cause a problem that degrades the usability of the user's 360 image. An example for solving this problem will be described with reference to FIG. 12.

12 is a diagram for explaining a concept in which advertisement content is disposed in a 360 image in real time outside the user's field of view according to an embodiment of the present invention.

Referring to FIG. 12, the current field of view may be the first partial image 10-1 as shown in FIG. 12. Of course, the field of view may not be limited to a partial image unit, and a part of the second partial image 11-1 and a part of the sixth partial image 15-1 may be included in the field of view.

According to an example of the present invention, the system (eg, the control unit 110) includes an advertisement area (eg, a third partial image (eg, a third part image) among areas excluding the area currently displayed on the user's field of view, that is, the terminal 100 of the user. 12-1), the fourth part image (13-1), and the fifth part image (14-1) may be arranged advertising content (eg, 50, 51, 52).

At this time, a notification signal (for example, a predetermined sound or display) indicating that the advertisement content is disposed outside the user's field of view may be output to allow the user to recognize that the advertisement content is disposed.

In the end, the advertisement method according to the technical idea of the present invention is a problem in which a specific advertisement content is pre-arranged in a predetermined image covering an angle of view even if it is not a 360 image or 360 degrees, and the user has never confirmed what the original image is. Can solve it.

In addition, while the user is consuming the image, there is an effect that the advertisement placement can be performed in real time, and the placement location of the advertisement can also search for an effective placement location so that the advertisement effect is excellent or does not significantly degrade the usability of the original image. It has an effect.

The method for generating a matched image or advertising method according to an embodiment of the present invention may be implemented as a computer readable code on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data readable by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tapes, hard disks, floppy disks, and optical data storage devices. In addition, the computer-readable recording medium can be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. In addition, functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers in the technical field to which the present invention pertains.

The present invention has been described with reference to one embodiment shown in the drawings, but this is merely exemplary, and those skilled in the art will understand that various modifications and other equivalent embodiments are possible therefrom. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

Claims (12)

Determining, by the system, the starting area identified by the user among the 360 images;
Adding the display area determined by the system as an advertising area in which predetermined advertising content can be placed; And
A method of providing advertisement using a 360 image, the method comprising the step of placing predetermined advertisement content in the current advertisement area by the system,
The advertisement providing method using the 360 image,
Determining a placement location in which the advertisement content is to be placed in the advertisement area; And
The method further includes disposing the advertisement content at the determined placement location,
In the advertisement area, determining a placement location where the advertisement content is to be placed,
A method of providing advertisement using a 360 image for determining a non-critical region in which the advertisement content can be displayed-the non-critical region is determined based on image complexity.
The method of claim 1, wherein the 360 image,
N partial images corresponding to a certain angle and facing in different directions are registered images,
Advertisement method using a 360 image, characterized in that the user-identified display area is determined by the partial image unit.
delete delete The method of claim 1, wherein the 360 image,
N images corresponding to a certain angle and directed in different directions (where n is an integer of 2 or more) are matched images,
In the advertisement area, determining a placement location where the advertisement content is to be placed,
Advertisement using 360 images, characterized in that before starting to shoot the k-th partial image among the partial images (k is an integer from 2 to n), the placement position included in the (k-1) partial image is started. How to provide.
Determining, by the system, the starting area identified by the user among the 360 images;
Adding the display area determined by the system as an advertising area in which predetermined advertising content can be placed; And
The system comprises the step of placing a predetermined advertisement content in the current advertisement area,
The step of the system to place a predetermined advertisement content in the current advertisement area,
Disposing the advertisement content in an advertisement area not currently included in the user's field of view in the advertisement area; And
And placing the advertisement content in an advertisement area not included in the user's field of view, and outputting a notification signal indicating that the advertisement content is disposed.
The system starting to display the 360 image on the user's terminal;
While the system is displaying the 360 image, placing predetermined advertisement content at a predetermined placement location included in the 360 image,
The step of disposing the advertisement content in a predetermined placement position included in the 360 image,
A method of providing advertisement using a 360 image, characterized in that the advertisement content is arranged in an area not included in the current field of view of the user.
Displaying, by the system, a matching image in which n partial images (where n is an integer greater than or equal to 2) corresponding to a certain angle and facing in different directions are matched; And
The system comprises the step of determining the placement location where the advertisement content will be placed,
The step of determining the placement position,
A method of providing advertisement, characterized in that before starting to shoot the k-th partial image among the partial images (k is an integer from 2 to n), the placement position included in the (k-1) partial image is started.
A computer program recorded in a computer-readable recording medium installed in a data processing apparatus and performing the method according to any one of claims 1, 2, or 5 to 8.
In the system,
A control unit including a processor; And
It includes a storage medium in which the program executed by the control unit is recorded,
The program,
Among the 360 images, the user determines the display area identified by the user, adds the determined display area as an advertisement area in which predetermined advertisement content can be placed, and places the predetermined advertisement content in the current advertisement area,
The program,
In the advertisement area, the placement location in which the advertisement content is to be placed is determined, and the advertisement content is placed in the determined placement location,
A system for providing an advertisement for determining the non-critical region in which the advertisement content can be displayed-the non-critical region is determined based on image complexity-as the placement location.
In the system,
A control unit including a processor; And
It includes a storage medium in which the program executed by the control unit is recorded,
The program,
While the system starts displaying the 360 image on the user's terminal and displays the 360 image, the advertisement content is placed at a predetermined placement location included in the 360 image,
The arrangement position,
A system for providing advertisements, characterized in that the area is not included in the current field of view of the user.
In the system,
A control unit including a processor; And
It includes a storage medium in which the program executed by the control unit is recorded,
The program,
Displaying a matched image in which n partial images corresponding to a certain angle and facing in different directions (where n is an integer of 2 or more) are matched, and determining a placement position where advertisement content will be placed,
In order to determine the placement position, starting to determine the placement position included in the (k-1) partial image before taking the kth partial image among the partial images (k is an integer from 2 to n). A system that provides advertisements characterized by.

Images