KR20210035638A - Image capturing device and method, computer-readable storage medium and computer program for controlling the image capturing device and image processing device and method, computer-readable storage medium and computer program for controlling the image processing device - Google Patents

Abstract

According to various embodiments, the present invention provides an image photographing device, which comprises: a camera module; a sensor module; a communication module; and a control unit acquiring one or more images including a subject by using one or more image sensors included in the camera module, checking a distance from the subject by using at least one sensor included in the sensor module, and controlling information corresponding to the distance from the subject and the acquired one or more images to be transmitted to an image processing device by using the communication module. The image photographing device can minimize resources required for comparing images, by setting a reference line for stitching when stitching the acquired images.

Description

An image capturing device, a method for controlling the image capturing device, a computer-readable recording medium and computer program, an image processing device and a method for controlling the image processing device, a computer-readable recording medium and a computer program TECHNICAL FIELD METHOD, COMPUTER-READABLE STORAGE MEDIUM AND COMPUTER PROGRAM FOR CONTROLLING THE IMAGE CAPTURING DEVICE AND IMAGE PROCESSING DEVICE AND METHOD, COMPUTER-READABLE STORAGE MEDIUM AND COMPUTER PROGRAM FOR CONTROLLING THE IMAGE PROCESSING DEVICE}

The present invention relates to an image photographing apparatus, a method for controlling the image photographing apparatus, a computer readable recording medium and a computer program, an image processing apparatus and a method for controlling the image processing apparatus, a computer readable recording medium and a computer program. will be.

With the recent commercialization of 5th generation mobile telecommunication (5G), a real-time live broadcast service that combines 360-degree camera and virtual reality (VR) technology as an immersive media application service and a high-quality on-demand video system ( Technology development is being made to provide content of video on demand (VOD).

Conventionally, when stitching images acquired using a 360-degree camera, an operator manually analyzes the overlapped image in a zone (also referred to as an overlapping zone) where some images overlap, and one overlapped image. Is discarded and stitching is in progress. However, in the case of such a stitching operation, there has been a problem in that the quality of the matched image varies depending on the operator's capability or the operator's condition.

In addition, conventional stitching technology has been developed in which a stitching device (stitching server) automatically analyzes the superimposed image from the acquired images and discards one superimposed image and performs stitching. However, the conventional stitching technology has a dark screen. Or, if the distance of the subject is moving, processing resources increase and precise matching is difficult.

Accordingly, an embodiment of the present invention provides an image capturing device providing a sophisticated stitching technology for images, a method for controlling the image capturing device, a computer-readable recording medium and a computer program, an image processing device, and the image processing device. It is possible to provide a method for performing, a computer-readable recording medium, and a computer program.

For example, when stitching acquired images, the image capturing apparatus can minimize resources required for comparing images by setting a reference line for stitching, and stitching operations at a fast response speed for moving subjects. Can run. In addition, the image capturing apparatus may perform a stitching operation according to the reference line, thereby enabling high-quality stitching even in a dark environment with low illuminance.

According to an embodiment, an image capturing device includes: a camera module; Sensor module; Communication module; And one or more images including a subject are obtained using one or more image sensors included in the camera module, and a distance to the subject is checked using at least one sensor included in the sensor module, and the communication module It may include a control unit that controls to transmit information corresponding to the distance to the subject and the acquired one or more images to the image processing apparatus by using.

According to an embodiment, an image processing apparatus includes: a communication module; And receiving information corresponding to a distance between the image capturing device and a subject and images obtained by the image capturing device from an image capturing device using the communication module, and based on the information corresponding to the received distance, the It may include a control unit that sets a reference line for stitching the received images.

According to an embodiment, a method of controlling an image capturing apparatus includes: acquiring one or more images including a subject using one or more image sensors; Checking a distance to the subject using at least one sensor; And transmitting information corresponding to the distance to the subject and the acquired one or more images to an image processing apparatus using a communication module.

According to an embodiment, a method of controlling an image processing device includes an operation of receiving information corresponding to a distance between the image capturing device and a subject from an image capturing device using a communication module and images acquired by the image capturing device. ; And setting a reference line for stitching the received images based on the information corresponding to the received distance.

According to an embodiment, there is provided a computer-readable recording medium storing a computer program, the method comprising: acquiring one or more images including a subject using one or more image sensors; Checking a distance to the subject using at least one sensor; And a command for causing the processor to perform a method including an operation of transmitting information corresponding to the distance to the subject and the acquired one or more images to an image processing apparatus using a communication module.

According to an embodiment, as a computer-readable recording medium storing a computer program, information corresponding to a distance between the image capturing device and a subject from an image capturing device using a communication module and an image acquired by the image capturing device Receiving them; And a command for causing the processor to perform a method including an operation of setting a reference line for stitching the received images based on the information corresponding to the received distance.

According to an embodiment, there is provided a computer program stored in a computer-readable recording medium, comprising: acquiring one or more images including a subject using one or more image sensors; Checking a distance to the subject using at least one sensor; And a command for causing the processor to perform a method including an operation of transmitting information corresponding to the distance to the subject and the acquired one or more images to an image processing apparatus using a communication module.

According to an embodiment, as a computer program stored in a computer-readable recording medium, information corresponding to a distance between the image capturing device and a subject from an image capturing device using a communication module and an image acquired by the image capturing device Receiving them; And a command for causing the processor to perform a method including an operation of setting a reference line for stitching the received images based on the information corresponding to the received distance.

An image photographing apparatus according to an embodiment of the present invention, a method for controlling the image photographing apparatus, a computer readable recording medium and a computer program, an image processing apparatus and a method for controlling the image processing apparatus, a computer readable recording medium, and Computer programs can provide sophisticated stitching techniques for images.

For example, when stitching acquired images, the image capturing apparatus can minimize resources required for comparing images by setting a reference line for stitching, and stitching operations at a fast response speed for moving subjects. Can run. In addition, the image capturing apparatus may perform a stitching operation according to the reference line, thereby enabling high-quality stitching even in a dark environment with low illuminance.

1 is a block diagram of a system including an image capturing apparatus and an image processing apparatus according to an embodiment of the present invention.
2A and 2B are diagrams illustrating an image capturing apparatus according to an exemplary embodiment of the present invention.
3A to 3D are views for explaining the operation of an image capturing apparatus and an image processing apparatus according to an embodiment of the present invention.
4 is a diagram for describing a stitching operation on images of an image capturing apparatus or an image processing apparatus according to an exemplary embodiment of the present invention.
5 is a flowchart of an operation of an image capturing apparatus according to an embodiment of the present invention.
6 is a flowchart of an operation of an image processing apparatus according to an embodiment of the present invention.
7 is a flowchart of an operation of an image capturing apparatus according to an embodiment of the present invention.

First, the advantages and features of the present invention, and a method of achieving them will become apparent with reference to embodiments to be described later in detail together with the accompanying drawings. Here, the present invention is not limited to the embodiments disclosed below, but may be implemented in a variety of different forms, and only these embodiments are intended to complete the disclosure of the present invention, and are common in the technical field to which the present invention pertains. It is provided by way of example in order for those skilled in the art to clearly understand the scope of the invention, so the technical scope of the invention should be defined by the claims.

In addition, in the following description of the present invention, if it is determined that a detailed description of known functions or configurations may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted. In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or custom of users or operators. Therefore, the definition should be made based on the technical idea described throughout the present specification.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a system including an image capturing apparatus and an image processing apparatus according to an embodiment of the present invention.

Referring to FIG. 1, the image capturing apparatus 100 may include a control unit 101, a camera module 103, a sensor module 105, a memory 107, and a communication module 109.

The control unit 101 (also referred to as a control device or a control circuit) includes at least one other component of the connected image capturing device 100 (eg, a hardware component (eg, a camera module 103, a sensor module 105), a memory device). (107) and the communication module 109) or software components) can be controlled, and various data processing and operations can be performed.

The camera module 103 may capture a still image and a video. For example, the camera module 103 may include one or more image sensors and one or more lenses. The lens may be a wide-angle lens or a telephoto lens. For example, the camera module 103 may be a 360 degree camera. The 360 degree camera may include a plurality of lenses and may capture images in a plurality of different directions.

The sensor module 105 may include one or more sensors capable of generating an electrical signal or data value corresponding to an internal operating state (eg, power) of the image capturing apparatus 100 or an external environmental state. For example, the sensor module 105 may include a distance measurement sensor. For example, the sensor module 105 may include one or more infrared ray sensors (IR Sensors) (also referred to as IR number/luminescence sensors). The infrared sensor may be used to check (detect, detect) a distance between the image capturing apparatus 100 and a subject.

The memory 107 includes various data used by at least one component (control unit 101, camera module 103, sensor module 105, and communication module 109) of the image capturing device 100, for example. For example, input data or output data for software (eg, a program) and commands related thereto may be stored. The memory 107 may include a volatile memory or a nonvolatile memory. For example, the memory 107 may store reference line information corresponding to each of specified distances between the imaging apparatus 100 and a subject for stitching images.

The communication module 109 may support establishing a wired or wireless communication channel between the image capturing apparatus 100 and the image processing apparatus 10 and performing communication through the established communication channel. For example, the communication module 109 is a wireless communication module (eg, a cellular communication module, a Wi-Fi communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) and/or a wired communication module (eg, a LAN (local area network) communication module or power line communication module), and short-range communication such as a first network (not shown) (e.g., Bluetooth, WiFi direct or IrDA (infrared data association)) using a corresponding communication module Network) or a second network (not shown) (eg, a cellular network, the Internet, or a telecommunication network such as a computer network (eg, LAN or WAN)).

According to an embodiment, the controller 101 acquires one or more images including a subject using one or more image sensors included in the camera module 103, and uses at least one sensor included in the sensor module. Thus, the distance to the subject is checked, and information corresponding to the distance to the subject and the acquired one or more images may be transmitted to the image processing apparatus 10 using the communication module 109.

For example, the one or more images may further include one or more other subjects, and in this case, the control unit 101 checks the distance to the one or more other subjects using the at least one sensor, and , It is confirmed that the distance between the image capturing device 100 and the subject is closer than the distance between the image capturing device 100 and the one or more other subjects, and the distance between the image capturing device 100 and the subject is the Information corresponding to the distance to the subject may be transmitted to the image processing apparatus 10 based on a distance between the image capturing apparatus 100 and the one or more other subjects.

For example, the one or more images may further include one or more other subjects, and in this case, the controller 101 checks the distance to the one or more other subjects using the at least one sensor, Information corresponding to the distance to the one or more other subjects may be transmitted to the image processing apparatus 10 by using the communication module.

The image processing apparatus 10 may include a control unit 11, a memory 17, and a communication module 19.

The control unit 11 (also referred to as a control device or a control circuit) includes at least one other component (for example, a hardware component (for example, a memory 17 and a communication module 19)) or software of the connected imaging device 100. Components) and perform various data processing and operations.

The memory 17 includes various data used by at least one component (control unit 11 and communication module 19) of the image processing apparatus 10, for example, software (eg, programs), and Input data or output data for related commands can be stored. The memory 17 may include a volatile memory or a nonvolatile memory. For example, the memory 17 may store reference line information corresponding to each of specified distances between the imaging apparatus 100 and a subject for stitching images.

The communication module 19 may support establishing a wired or wireless communication channel between the image processing apparatus 10 and the image capturing apparatus 100 and performing communication through the established communication channel.

According to an embodiment, the control unit 11 includes information corresponding to a distance between the image capturing device 100 and a subject from the image capturing device 100 using the communication module 19, and the image capturing device 100 ), and based on information corresponding to the received distance, a reference line for stitching the received images may be set.

For example, the controller 11 may set the reference line based on information corresponding to the received distance and reference line information stored in the memory.

For example, the controller 11 may stitch the received images based on the set reference line.

For example, the control unit 11 receives information corresponding to a distance between the image capturing device 100 and one or more other subjects from the image capturing device 100 using the communication module 19, and the Based on information corresponding to the distance between the image photographing apparatus 100 and the subject and information corresponding to the distance between the image photographing apparatus 100 and the one or more other subjects, It is confirmed that the distance is closer than the distance between the image capturing device 100 and the one or more other subjects, and the distance between the image capturing device 100 and the subject is the image capturing device 100 and the one or more other subjects The reference line may be set based on a distance closer than the distance between the two.

In the embodiment of FIG. 1 described above, it has been described that the image capturing device 100 and the image processing device 10 are configured as separate devices, but according to another embodiment, the image capturing device 100 and the image processing device 10 ) May be configured as one device. For example, the image capturing apparatus 100 may include a control unit 101, a camera module 103, a sensor module 105, a memory 107, and a communication module 109, as described above, In addition, the control unit 101 of the image capturing device 100 may perform at least one of the control unit 11 of the image processing device 10 described above so that the image capturing device 100 can execute the operation of the image processing device 10. You can also execute the control operation of.

In addition, in the embodiment of FIG. 1 described above, it has been described that the image processing device 10 stitches images, but according to another embodiment, stitching that is communicatively connected with the image capturing device 100 or the image processing device 10 The images may be stitched on a device (not shown). For example, the stitching apparatus may receive the set reference line and the images, and generate a stitched image by stitching the received images based on the set reference line.

Further, although not shown, the image capturing device 100, the image processing device 10, or a display device (not shown) connected in communication with the stitching device may receive and output the stitched image.

2A and 2B are diagrams illustrating an image capturing apparatus according to an exemplary embodiment of the present invention.

2A and 2B, the image capturing apparatus 100 may include a frame 201, a first support 207, and a second support 209.

The frame 201 (also referred to as a housing) may be a 360 degree camera including a plurality of lenses 203 and a plurality of image sensors 204 to generate a plurality of images. For example, as shown in FIGS. 2A and 2B, a plurality of lenses 203 may be disposed at predetermined intervals. The plurality of lenses 203 may be a wide-angle lens or a telephoto lens. For example, as shown in FIG. 2B, a plurality of image sensors 204 may be positioned (arranged) at a distance from the plurality of lenses 203.

The frame 201 may include one or more infrared sensors 205. For example, one or more infrared sensors 205 may be positioned (placed) between image sensors 204 as shown in FIGS. 2A and 2B.

The first support 207 (also referred to as a mount) may be attached to a portion of the frame 201. For example, the first support 207 may include one or more motors (not shown), and may move (rotate) to the left or right around a designated rotation axis (not shown) according to the rotation of the one or more motors. have.

The second support part 209 may be attached to a part (eg, the lower end) of the first support part 207. For example, the second support 209 may be in the form of a tripod, and various forms may be applied.

Referring to FIG. 2B, the image capturing apparatus 100 may check (detect) a subject and a distance to the subject through at least one of one or more infrared sensors 205. For example, the image capturing apparatus 100 may perform distance scanning on the subject using infrared rays 208 emitted through at least one of the one or more infrared sensors.

In the above-described embodiments of FIGS. 2A and 2B, it has been described that the image photographing apparatus 100 includes one or more infrared sensors to check a subject and a distance to the subject, but according to another embodiment, an image photographing apparatus Reference numeral 100 may include one or more conventional distance measuring sensors capable of checking the distance to the subject, and may check (detection, measure) the subject and the distance between the subject.

3A to 3D are views for explaining the operation of an image capturing apparatus and an image processing apparatus according to an embodiment of the present invention.

3A and 3B, the image capturing apparatus 100 may acquire (photograph) images corresponding to the subject 306 through the first image sensor 302 and the second image sensor 304. .

For example, the image acquired through the first image sensor 302 may be a first image 312 including an image corresponding to the subject 306 as shown in (a) of FIG. 3B, and the second image The image acquired through the sensor 302 may be a second image 314 including an image corresponding to the subject 306 as shown in (b) of FIG. 3B. The first image 312 and the second image 314 may have a circular shape as shown in FIGS. 3B (a) and (b).

According to an embodiment, the image capturing device 100 may check a distance between the image capturing device 100 and the subject 306 using the first infrared sensor 308.

For example, one or more designated infrared sensors corresponding to (related to) each of the plurality of image sensors of the imaging apparatus 100 may be set, and accordingly, at least one of the plurality of image sensors may be When an image corresponding to a subject is acquired, one or more designated infrared sensors corresponding to the at least one image sensor may be automatically activated. For example, when the first image sensor 302 and/or the second image sensor 304 acquires (shoots) an image corresponding to the subject 306 (when confirming the shooting of the subject 306) ), the image capturing apparatus 100 may activate a designated first infrared sensor 308 corresponding to the first image sensor 302 and/or the second image sensor 304, and the activated first infrared ray Using the sensor 308, the distance between the image capturing apparatus 100 and the subject 306 may be checked.

According to an embodiment, the image capturing device 100 or an image processing device (for example, the image processing device 10) communicating with the image capturing device 100 includes the identified image capturing device 100 and a subject ( Based on the distance between the images 306, a reference line (also referred to as a stitching line) for stitching the images acquired by the image capturing apparatus 100 may be set.

For example, the image capturing device 100 or the image processing device may store reference line information corresponding to each of specified distances between the image capturing device for stitching images and a subject in a memory. The image capturing device 100 or the image processing device may set a reference line based on the identified distance between the image capturing device 100 and the subject 306 and reference line information stored in the memory.

According to another embodiment, the image capturing device 100 or the image processing device includes a reference line for stitching the images acquired by the image capturing device 100 based on infrared rays emitted by an infrared sensor detecting a subject. Can be set.

For example, the image capturing device 100 or the image processing device may provide a first infrared sensor 308 for each of the first image 312 and the second image 314 acquired by the image capturing device 100. Reference lines 316 and 318 corresponding to the infrared rays 310 emitted by) may be generated. For example, the image corresponding to the infrared ray 310 included in the image acquired by the first image sensor 302 and the second image sensor 304 of the image capturing apparatus 100 is used as a reference (center). Lines 316 and 318 can be created. For example, the first image 312 and the second image 314 acquired by the first image sensor 302 and the second image sensor 304 are shown in FIGS. 3B (a) and (b). It may include a circle such as, and each of the reference lines 316 and 318 of the first image 312 and the second image 314 may have a parabolic shape. The parabola of the first image 312 indicates a reference point in an image corresponding to the upper and lower sides of the first image 312 and the infrared 310 when the center of the first image 312 is referenced. It can be created by connecting. The parabola of the second image 314 indicates a reference point in the image corresponding to the upper side and the lower side of the second image 314 and the infrared ray 310 when the center of the second image 314 is referenced. It can be created by connecting.

According to an embodiment, the image capturing apparatus 100 or the image processing apparatus may stitch the acquired images based on a reference line set for stitching the acquired images.

For example, the image capturing device 100 or the image processing device clips a part of the first image 312 based on the reference line 316 of the first image 312 and A part of the second image 314 is clipped based on the reference line 318 of 314, and the remaining part of the first image 312 and the remaining part of the second image 314 are stitched to FIG. 3B. The stitching image as shown in (c) of can be generated.

According to an embodiment, the generated stitching image may be output through the image capturing device 100 or a display device (not shown) connected to the image processing device.

Referring to FIGS. 3C and 3D, while the image capturing apparatus 100 acquires images corresponding to the subject 306 through the first image sensor 302 and the second image sensor 304, another subject ( According to the approach of 309), images corresponding to the other subject 309 may be acquired.

For example, the image acquired through the first image sensor 302 is a third image 320 including an image corresponding to the subject 306 and other subject 309 as shown in FIG. 3D(a). The image acquired through the second image sensor 302 may be a fourth image 322 including an image corresponding to the subject 306 and other subject 309 as shown in FIG. 3D(b). have. The third image 320 and the fourth image 322 may have a circular shape as shown in FIGS. 3D (a) and (b).

For example, when the other subject 309 approaches with the image capturing device 100, referring to FIG. 3C, the set reference line 316 of the first image 312 and the set reference of the second image 314 When the line 318 is not changed, the image of the other subject 309 is displayed by the set reference line 316 of the first image 312 and the set reference line 318 of the second image 314 May not be output through the device. In order to compensate for this point, while acquiring images corresponding to the subject 306 through the first image sensor 302 and the second image sensor 304, the image photographing apparatus 100 or the image processing apparatus, When another subject 309 approaches, the set reference lines 316 and 318 may be changed.

According to an embodiment, the image capturing device 100 may check a distance between the image capturing device 100 and the other subject 309 using the first infrared sensor 308.

According to an embodiment, the image capturing device 100 or the image processing device is obtained by the image capturing device 100 based on the identified distance between the image capturing device 100 and the other subject 309. A reference line (also referred to as a stitching line) for stitching one images may be reset. For example, when the distance between the imaging apparatus 100 and the other subject 309 is closer (shorter distance) than the distance between the imaging apparatus and the subject 306, the reference line may be reset. .

For example, the image capturing device 100 or the image processing device may reset the reference line based on the identified distance between the image capturing device 100 and another subject 309 and reference line information stored in a memory. .

According to another embodiment, the image capturing device 100 or the image processing device includes a reference line for stitching the images acquired by the image capturing device 100 based on infrared rays emitted by an infrared sensor detecting a subject. Can be reset.

For example, the image capturing device 100 or the image processing device may provide a first infrared sensor 308 for each of the third image 320 and the fourth image 322 acquired by the image capturing device 100. Reference lines 3324 and 324 corresponding to the infrared rays 310 emitted by) may be generated. Since the description of generating the reference line based on the infrared rays emitted by the infrared sensor has been described above, it will be omitted.

According to an embodiment, the image capturing apparatus 100 or the image processing apparatus may stitch the acquired images based on a reference line reset to stitch the acquired images.

For example, the image capturing apparatus 100 or the image processing apparatus clipping a part of the third image 320 based on the reset reference line 324 of the third image 320 and 4 A part of the fourth image 322 is clipped based on the reset reference line 326 of the image 322, and the remaining part of the third image 320 and the remaining part of the fourth image 322 are removed. A new stitched image can be created by stitching.

According to an embodiment, the generated new stitched image may be output through the image capturing device 100 or a display device (not shown) connected to the image processing device.

4 is a diagram for describing a stitching operation on images of an image capturing apparatus or an image processing apparatus according to an exemplary embodiment of the present invention.

According to an embodiment, the image capturing device 100 or the image processing device may store reference line information corresponding to each of specified distances between an image capturing device for stitching images and a subject in a memory.

For example, the image capturing device 100 or the image processing device may store reference line information corresponding to each of specified distances between the image capturing device for stitching images and a subject in a memory in the form of a lookup table. Can be saved.

For example, the image capturing device 100 or the image processing device may store pixel information corresponding to the reference line for generating a reference line for each distance between the image capturing device and a subject.

Referring to FIGS. 4A and 4B, according to the above-described embodiment of FIGS. 3A to 3D, when it is determined that the distance between the image photographing apparatus 100 and the subject 306 is 10 m, the image photographing apparatus (100) Alternatively, the image processing apparatus may check pixel information corresponding to the reference line for generating the reference line 318 corresponding to the 10m stored in the memory. The image capturing apparatus 100 or the image processing apparatus may generate a reference line 318 based on the identified pixel information.

When it is determined that the distance between the image capturing device 100 and the other subject 309 is 5m, the image capturing device 100 or the image processing device generates a reference line 326 corresponding to the 5m stored in the memory. It is possible to check pixel information corresponding to the reference line for. The image capturing apparatus 100 or the image processing apparatus may generate a reference line 326 based on the identified pixel information.

5 is a flowchart of an operation of an image capturing device (eg, the image capturing device 100 or the control unit 101 of the image capturing device 100) according to an embodiment of the present invention.

In operation 510, the image capturing apparatus may acquire one or more images including the subject by using one or more image sensors included in the camera module.

For example, the camera module may be a 360 degree camera.

In operation 520, the image capturing apparatus may check the distance to the subject using at least one sensor included in the sensor module.

For example, the at least one sensor may include an infrared sensor.

In operation 530, the image capturing apparatus may transmit information corresponding to the distance to the subject and the acquired one or more images to an image processing apparatus (eg, the image processing apparatus 10) using a communication module.

6 is a flowchart of an operation of an image processing device (eg, the image processing device 10 or the control unit 11 of the image processing device 10) according to an embodiment of the present invention.

In operation 610, the image processing device may receive information corresponding to the distance between the image capturing device and the subject from the image capturing device using a communication module and images acquired by the image capturing device.

In operation 620, the image processing apparatus may set a reference line for stitching the received images based on information corresponding to the received distance.

For example, the image capturing device may set a reference line based on the identified distance between the image capturing device and the reference object and reference line information stored in a memory.

According to the above-described embodiments of FIGS. 5 and 6, an image capturing device acquires one or more images, checks a distance between a subject included in the image and the image capturing device, and allows the image processing device to stitch images. Although it has been described as setting a reference line, according to another embodiment, the image capturing apparatus may perform an operation of setting a reference line for stitching images of the image processing apparatus.

7 is a flowchart of an operation of an image capturing device (eg, the image capturing device 100 or the control unit 101 of the image capturing device 100) according to an embodiment of the present invention.

In operation 710, the image capturing apparatus may acquire images including a plurality of subjects.

According to an embodiment, one or more image sensors included in a camera module of the image capturing apparatus may acquire images including a plurality of subjects. For example, the camera module may capture an image (movie) in real time.

According to an embodiment, while acquiring images including the first subject, the image capturing apparatus may acquire images including the first subject and the second subject according to the entry of the second subject.

For example, when the image capturing device acquires images including the first subject, the image capturing device or the image processing device communicating with the image capturing device may include the first A first reference line corresponding to the subject may be set. In a state in which the first reference line corresponding to the first subject is set, according to the entry of the second subject, the image capturing apparatus may acquire images including the first subject and the second subject.

In operation 720, the image capturing apparatus may select a reference subject for setting a reference line for stitching the acquired images from among the plurality of subjects.

According to an embodiment, the image capturing device may select the reference subject based on a distance between the image capturing device and each of the plurality of subjects.

For example, by checking the distance between the image capturing device and each of the plurality of subjects, a subject located closest to the image capturing device may be selected as the reference subject. For example, while the image capturing apparatus acquires images including the first subject, when the second subject is located closer to the image capturing device than the first subject according to the entry of the second subject, the The second subject may be selected as the reference subject.

In operation 730, the image capturing apparatus may set a reference line for stitching the acquired images based on the distance to the reference subject.

According to an embodiment, the image capturing device may set a reference line based on the identified distance between the image capturing device and the reference subject and reference line information stored in a memory.

According to another embodiment, the imaging device may reset a reference line for stitching images acquired by the imaging device based on infrared rays emitted by an infrared sensor that detects a subject.

In addition to the above-described embodiment of FIG. 7, the image capturing apparatus may generate a stitched image by stitching the acquired images based on (as a reference) the set reference line. Alternatively, the image capturing device may be communicatively connected with an image processing device or a stitching device, and corresponding information (acquired images and Information corresponding to the reference line) may also be transmitted.

In addition, in the embodiment of FIG. 7 described above, it has been described that the selection of the reference subject is based on the distance between the image capturing device and the subject, but according to another embodiment, the image capturing device is based on a user input through an input device or the like. One of the plurality of subjects may be selected as a reference subject.

In addition, in the embodiment of FIG. 7 described above, it has been described that the image capturing device sets a reference line for stitching images, but according to another embodiment, the image capturing device includes the acquired images and the image capturing device. And information corresponding to the distance to the reference subject may be transmitted to the image processing apparatus, so that the image processing apparatus may set a reference line for stitching the images.

Various embodiments of the present document include instructions stored in a machine-readable storage media (eg, memory 113 (internal memory or external memory)) that can be read by a machine (eg, a computer). It can be implemented in software (eg, a program). The device is a device capable of calling a stored command from a storage medium and operating according to the called command, and may include an electronic device (eg, the electronic device 100) according to the disclosed embodiments. When the command is executed by a control unit (eg, the control unit 101) (or a processor), the control unit may perform a function corresponding to the command directly or by using other components under the control of the control unit. Instructions may include code generated or executed by a compiler or interpreter. A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here,'non-transient' means that the storage medium does not contain a signal and is tangible, but does not distinguish between semi-permanent or temporary storage of data in the storage medium.

According to an embodiment, a method according to various embodiments disclosed in the present document may be provided by being included in a computer program product.

According to an embodiment, as a computer-readable recording medium storing a computer program, an operation of acquiring one or more images including a subject using one or more image sensors, and a distance to the subject using at least one sensor Including an instruction for causing the processor to perform a method including an operation of checking and transmitting information corresponding to a distance to the subject and an operation of transmitting the acquired one or more images to an image processing apparatus using a communication module. I can.

According to an embodiment, as a computer-readable recording medium storing a computer program, information corresponding to a distance between the image capturing device and a subject from an image capturing device using a communication module and an image acquired by the image capturing device Including an instruction for causing the processor to perform a method including an operation of receiving signals and setting a reference line for stitching the received images based on information corresponding to the received distance. I can.

According to an embodiment, as a computer program stored in a computer-readable recording medium, an operation of acquiring one or more images including a subject using one or more image sensors, and a distance to the subject using at least one sensor Including an instruction for causing the processor to perform a method including an operation of checking and transmitting information corresponding to a distance to the subject and an operation of transmitting the acquired one or more images to an image processing apparatus using a communication module. I can.

According to an embodiment, as a computer program stored in a computer-readable recording medium, information corresponding to a distance between the image capturing device and a subject from an image capturing device using a communication module and an image acquired by the image capturing device Including an instruction for causing the processor to perform a method including an operation of receiving signals and setting a reference line for stitching the received images based on information corresponding to the received distance. I can.

The above description is merely illustrative of the technical idea of the present invention, and those of ordinary skill in the technical field to which the present invention pertains, various substitutions, modifications and changes, etc., within the scope not departing from the essential characteristics of the present invention. It will be easy to see that this is possible. That is, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain the technical idea, and the scope of the technical idea of the present invention is not limited by these embodiments.

Accordingly, the scope of protection of the present invention should be interpreted by the claims to be described later, and all technical thoughts within the scope equivalent thereto should be construed as being included in the scope of the present invention.

According to an embodiment of the present invention, an image capturing apparatus may be applied and used in a technical field in which images are acquired and stitched using a 360 degree camera to generate a stitched image.

Claims (12)

Camera module;
Sensor module;
Communication module; And
Acquiring one or more images including a subject using one or more image sensors included in the camera module,
Checking the distance to the subject using at least one sensor included in the sensor module,
And a control unit for controlling to transmit information corresponding to a distance to the subject and the acquired one or more images to an image processing device using the communication module. The method of claim 1, wherein the at least one sensor,
An imaging device including an infrared sensor. The method of claim 1, wherein the camera module,
A video recording device that is a 360-degree camera. The method of claim 1, wherein the one or more images,
It further includes one or more other subjects,
The control unit,
Checking the distance to the one or more other subjects using the at least one sensor,
Confirming that the distance between the image photographing device and the subject is closer than the distance between the image photographing device and the one or more other subjects,
An image photographing apparatus for transmitting information corresponding to the distance to the subject to the image processing apparatus based on the fact that the distance between the image photographing apparatus and the subject is closer than the distance between the image photographing apparatus and the one or more other subjects. The method of claim 1, wherein the one or more images,
It further includes one or more other subjects,
The control unit,
Checking the distance to the one or more other subjects using the at least one sensor,
An image capturing device further transmitting information corresponding to a distance to the one or more other subjects to the image processing device by using the communication module. Communication module; And
Information corresponding to the distance between the image capturing device and the subject and images acquired by the image capturing device are received from an image capturing device using the communication module, and the reception is based on the information corresponding to the received distance. An image processing apparatus comprising a control unit for setting a reference line for stitching the images. The method of claim 6,
Further comprising a memory for storing reference line information corresponding to each of the specified distances between the image capturing device and a subject for stitching images,
The control unit,
An image processing apparatus configured to set the reference line based on information corresponding to the received distance and reference line information stored in the memory. The method of claim 6, wherein the control unit,
An image processing apparatus for stitching the received images based on the set reference line. The method of claim 6, wherein the control unit,
Receiving information corresponding to the distance between the image capturing device and one or more other subjects from the image capturing device using the communication module,
Based on information corresponding to the distance between the image photographing device and the subject and information corresponding to the distance between the image photographing device and the one or more other subjects, the distance between the image photographing device and the subject is determined by the Confirm that it is closer than the distance between one or more other subjects,
An image capturing apparatus configured to set the reference line based on a distance between the image capturing device and the subject being closer than a distance between the image capturing device and the one or more other subjects. Receiving information corresponding to a distance between the image capturing device and a subject and images acquired by the image capturing device from an image capturing device using a communication module; And
And setting a reference line for stitching the received images based on information corresponding to the received distance. As a computer-readable recording medium storing a computer program,
The computer program,
A computer-readable recording medium comprising instructions for causing a processor to perform the method according to claim 10. As a computer program stored in a computer-readable recording medium,
The computer program,
A computer program comprising instructions for causing a processor to perform the method according to claim 10.

Images