WO2020101072A1

WO2020101072A1 - Device and method for generating and storing digital note

Info

Publication number: WO2020101072A1
Application number: PCT/KR2018/014012
Authority: WO
Inventors: 박용식; 허문혁
Original assignee: 망고슬래브 주식회사
Priority date: 2018-11-14
Filing date: 2018-11-15
Publication date: 2020-05-22
Also published as: KR102075847B1

Abstract

The present disclosure relates to a device and a method for generating a digital note and, more specifically, to a device and a method for generating a digital note which generates a digital note image by photographing the upper surface of a note paper holder capable of accommodating note paper with at least one focus at an angle inclined with respect to the upper surface and synthesizing one or more photographed images, such that a user can digitize and store a note written on paper or adhesive note paper, a so-called analog note, and thus can easily manage and utilize the note generated by the user while maintaining the convenience and feeling of handwriting a note.

Description

Digital memo creation and storage device and method

The present disclosure relates to an apparatus and method for generating and storing a digital memo, and more particularly, to an apparatus and method for generating and storing an image of a memo using a camera.

In general, a memo is a short article or picture that records or summarizes new ideas in order not to forget something that needs to be remembered during daily life or work, most of which are read or conveyed to others. Write when you need to do it, or remember important points under review and discussion. In particular, if you write down important or important items in the process of drawing new ideas or finding answers to solving complex problems, you can increase work or academic efficiency.

For this reason, many people have a habit of taking notes in the course of daily life, work, or learning, and take notes using writing instruments such as pencils or pens on small-sized paper or adhesive notes. However, it is possible to check and revisit a note written directly on a paper or an adhesive note by pasting it on a wall or white board, etc., but it may require a lot of space for storing the note and may be lost later. In addition, in order to share the contents of the memo with multiple people, there is a hassle of having to reproduce or copy to another memo paper.

On the other hand, due to the conviction of portable computing devices such as smart phones, the number of digital memos stored in the portable computing devices using a touch pen or keyboard is increasing. Such a digital memo is relatively easy to store and store than an analog memo, which is a memo on a memo paper, but it is not easy to search a memo desired by the user.

In addition, dedicated products are being released that can create digital memos. However, since these digital memo products require separate input devices such as a dedicated pen and a dedicated note pad, they are relatively expensive and have a poor writing feel compared to those written on regular memo papers. It does not meet the needs of the user who wants to save. Accordingly, despite the development of digital memo generation and storage technology, many users still prefer to write in a memo paper in a manner that is excellent in convenience and writing and is familiar with existing memo habits.

In the embodiments disclosed in the present specification, a memo created on a paper or an adhesive memo, aka analog memo, is digitally stored and stored, so that the memo created by the user can be easily managed and easily utilized while maintaining the convenience and handwriting of the memo. It is intended to provide an apparatus and method.

A digital memo generating apparatus according to an embodiment of the present disclosure includes a paper holder capable of accommodating a memo paper; A camera including an optical system and an image sensor arranged to photograph the upper plane of the paper holder with at least one focus at an angle inclined to the upper plane; And a processor generating a digital memo image by synthesizing one or more images taken by the camera.

In the digital memo generation apparatus according to an embodiment of the present disclosure, the optical system of the camera may be arranged such that the optical axis of the optical system has an angle of less than 90 degrees with respect to the upper plane of the paper holder.

The digital memo generating apparatus according to an embodiment of the present disclosure may include the lens configured to have at least one focus with respect to the upper plane of the paper holder.

In the digital memo generating apparatus according to an embodiment of the present disclosure, the image sensor of the camera has an imaging surface arranged parallel to the upper plane of the paper holder, and the optical system of the camera comprises: It may include a mirror arranged to reflect the light incident from the upper plane to be irradiated to the imaging surface of the image sensor.

In the digital memo generating apparatus according to an embodiment of the present disclosure, the processor may synthesize the one or more images by calculating an average value for each pixel of the one or more images.

The digital memo generation apparatus according to an embodiment of the present disclosure may perform distortion compensation of the synthesized image by applying a perspective projection transformation to the synthesized image.

The digital memo generating apparatus according to an embodiment of the present disclosure may further include a communication unit that transmits the digital memo image to an external device.

The digital memo generation apparatus according to an embodiment of the present disclosure may further include an input unit that receives a user input driving the camera and the processor to generate the digital memo image.

A digital memo generating apparatus according to another embodiment of the present disclosure includes a paper holder capable of accommodating a memo paper; And an optical system and an image sensor arranged to photograph the upper plane of the paper holder with at least one focus at an angle inclined with respect to the upper plane, wherein the camera comprises: the upper plane of the paper holder. The processor may be configured to generate a digital memo image by synthesizing the one or more images by transmitting the captured one or more images to a processor.

In the digital memo generation apparatus according to another embodiment of the present disclosure, the optical system of the camera may be arranged such that the optical axis of the optical system has an angle of less than 90 degrees with respect to the upper plane of the paper holder.

The digital memo generating apparatus according to another embodiment of the present disclosure may include the lens configured to have at least one focus with respect to the upper plane of the paper holder.

In a digital memo generating apparatus according to another embodiment of the present disclosure, the image sensor of the camera has an imaging surface arranged parallel to the upper plane of the paper holder, and the optical system of the camera comprises: the upper portion of the holder It may include a mirror arranged to reflect the light incident from the plane to be irradiated to the imaging surface of the image sensor.

The digital memo generating apparatus according to another embodiment of the present disclosure may further include a communication unit transmitting the image to the processor.

The digital memo generating apparatus according to another embodiment of the present disclosure may further include an input unit that receives a user input driving the camera to photograph the image.

The method for generating a digital memo according to an embodiment of the present disclosure includes: by the camera arranged to photograph the upper plane of the paper holder accommodating the note paper with at least one focus at an angle inclined to the upper plane, the paper Taking one or more images of the upper plane of the holder; And generating, by the processor, a digital memo image by synthesizing the one or more images captured by the camera.

A method for generating a digital memo according to an embodiment of the present disclosure includes: the camera, an optical system arranged to photograph the upper plane of the paper holder with at least one focus at an angle inclined to the upper plane, and the paper And an image sensor having an imaging surface arranged parallel to the upper plane of the holder, the method reflecting light incident from the upper plane of the paper holder by the optical system of the camera and irradiating the image sensor It may further include a step.

A method for generating a digital memo according to an embodiment of the present disclosure, wherein the generating of the digital memo image includes synthesizing the one or more images by calculating an average value for each pixel of the one or more images by the processor. can do.

In the method of generating a digital memo according to an embodiment of the present disclosure, the step of generating the digital memo image may include applying the perspective projection transformation to the synthesized image by the processor. It may include performing a distortion compensation of.

The method for generating a digital memo according to an embodiment of the present disclosure may further include transmitting, by the communication unit, the synthesized image to an external device.

A non-transitory computer-readable storage medium storing instructions for digital memo generation according to an embodiment of the present disclosure, when the instructions are executed by a processor, cause the processor to open an upper plane of the paper holder that receives the note paper. Photographing one or more images of an upper plane of the paper holder by a camera arranged to shoot with at least one focus at an angle inclined with respect to the upper plane; And synthesizing one or more images captured by the camera to generate a digital memo image.

According to various embodiments of the present disclosure, by managing a memo written by a user with a camera and generating and storing a digital memo, it is possible to increase the management and utilization of the memo. In addition, the digital memo generated as described above can be conveniently output to a memo printer or transmitted to another computer or a memo cloud service system to conveniently manage and recycle the digital memo.

The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will become apparent to those skilled in the art from the description of the claims.

Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, where similar reference numerals indicate similar elements, but are not limited thereto.

1 is a view showing a digital memo generating device according to an embodiment of the present disclosure.

2 is a view showing the configuration of the camera and the paper holder of the digital memo generating device according to an embodiment of the present disclosure.

3 is a block diagram showing the configuration of a digital memo generating device according to an embodiment of the present disclosure.

4 is a diagram illustrating a detailed configuration of a camera of a digital memo generating device according to an embodiment of the present disclosure.

5 is a diagram illustrating a digital memo storage system including a digital memo generation device and other devices connected to a network according to an embodiment of the present disclosure.

6 is a diagram illustrating a digital memo storage system including a digital memo generating device and other devices connected to a network according to another embodiment of the present disclosure.

7 is a view showing the configuration of a camera and a paper holder of a digital memo generating device according to another embodiment of the present disclosure.

8 is a view showing the configuration of a camera and a paper holder of a digital memo generating device according to another embodiment of the present disclosure.

9 is a flowchart illustrating a digital memo generation method according to an embodiment of the present disclosure.

10 is a diagram illustrating a procedure of synthesizing a plurality of memo images by a digital memo generation method according to an embodiment of the present disclosure.

11 is a diagram illustrating a focus area detection method according to an embodiment of the present disclosure.

12 is a diagram illustrating a method of synthesizing a plurality of memo images according to the size of a frequency component according to an embodiment of the present disclosure.

13 is a diagram illustrating a method of cropping and combining regions in focus in a plurality of images according to an embodiment of the present disclosure.

14 is a diagram illustrating a procedure for compensating distortion of a memo image by a digital memo generation method according to an embodiment of the present disclosure.

15 is a flowchart illustrating a method for generating and storing a digital memo according to another embodiment of the present disclosure.

Hereinafter, with reference to the accompanying drawings, specific details for the practice of the present disclosure will be described in detail. However, in the following description, when there is a risk of unnecessarily obscuring the subject matter of the present disclosure, detailed descriptions of well-known functions or configurations will be omitted.

In the accompanying drawings, identical or corresponding components are given the same reference numerals. In addition, in the following description of the embodiments, it may be omitted to describe the same or corresponding elements overlapping. However, although descriptions of components are omitted, it is not intended that such components are not included in any embodiment.

1 is a view showing a digital memo generating device according to an embodiment of the present disclosure, Figure 2 is a view showing the configuration of a camera and a paper holder of the digital memo generating device according to an embodiment of the present disclosure, Figure 3 is It is a block diagram showing the configuration of a digital memo generating device according to an embodiment of the present disclosure. 4 is a diagram illustrating a detailed configuration of a camera of a digital memo generating device according to an embodiment of the present disclosure, and FIGS. 5 and 6 are other devices connected to a network with the digital memo generating device according to an embodiment of the present disclosure It is a view showing a digital memo storage system including the.

1 and 2, the digital memo generating apparatus 100 includes a memo pad 110 having a structure in which a memo paper 190 such as one or more general memo papers or adhesive memo papers can be stacked and accommodated. Includes. The memo pad 110 may be made of various materials, such as plastic resin, wood, metal, and the like, and may be formed in various forms.

As an example of the memo pad 110, as shown in FIG. 1, a paper holder 116 having a concave portion and an opening may be formed in the memo pad lower portion 114, the concave portion of the memo pad lower portion 114. One or more sheets of memo paper 190 may be stacked and accommodated as the upper surface is impregnated downward, and the opening is a part or all of the sidewalls forming the recess and is partially or completely opened to the recess through the opening. The stacked and received note paper 190 can be easily gripped and pulled out one by one or several sheets.

In the present disclosure, the top surface of the opening (that is, the side on which the memo paper 190 is inserted) may be referred to as a “top plane” or a “top side”. Here, the "top plane" or "upper side" of the memo pad 110 may be arranged substantially parallel to the top surface of the memo paper 190 accommodated by being stacked on the memo pad 110.

The memo pad upper part 112 may be integrally formed with the memo pad lower part 114 or separated and combined, and may be formed to have a predetermined height from the upper plane of the paper holder 116. In addition, the memo pad upper part 112 may be formed to form a predetermined angle with respect to the upper plane of the paper holder 116. At least a part of the memo pad upper part 112, in order to avoid interference by the memo pad upper part 112 in the process of inserting the memo paper 190 into the paper holder 116 or taking a memo on the memo paper 190 , May be formed to form an obtuse angle (b) with respect to the paper holder 116.

The memo pad 110 may include a camera 120 for photographing the memo paper 190 as shown in FIGS. 1 to 3. The camera 120 may include an optical system and an image sensor arranged to photograph the upper plane of the paper holder 160 with at least one focus at an angle inclined to the upper plane.

The optical system of the camera 120 may be arranged such that the imaging surface of the image sensor is inclined with respect to the top surface of the memo paper 190 accommodated in the paper holder 160 or the upper plane of the paper holder 160. In one embodiment, the optical axis of the optical system of the camera 120 may be arranged to have an angle a of less than 90 degrees with respect to the upper plane of the paper holder 160. Accordingly, the camera 120 may be provided on at least a portion of the memo pad top 112 formed to form an angle b of less than 90 degrees with respect to the paper holder 116.

In addition, the optical system of the camera 120 is configured such that the image sensor 122 can take the top plane of the paper holder 160 or the top surface of the memo paper 190 accommodated in the paper holder 160 with at least one focus. It may include a lens 124. For example, the lens 124 may be positioned such that the image sensor 122 has a first focus 126 and a second focus 128 as illustrated in FIG. 4. In this case, the lens 124 may be driven or controlled such that the first focus 126 of the image sensor 122 moves to the second focus 128. In one example, the lens 124 of the camera 120 may be moved by an AF actuator (Auto Focus Actuator).

Therefore, the digital memo generating device 100 when the user makes a memo using the writing instrument 192 such as a pencil, pen, etc. on the memo paper 190 accommodated in the paper holder 116, at least one or more through the camera 120 The memo paper 190 is photographed with focus to generate one or more images. In the present disclosure, “image” or “memo image” refers to the memo pad 190 received in the paper holder 116 through the camera 120 integrally or assembledly coupled to the memo pad 110 or the paper holder 116. It may refer to the image taken.

3, the memo pad 110 may include a processor 150 that generates a digital memo image from a synthesized image by synthesizing one or more images captured by the camera 120. In one example, the processor 150 may be embedded or mounted inside the memo pad 110. In another example, the processor 150 may be configured to be arranged independently of the memo pad 110, and in this case, may transmit and receive image data through the wireless or wired communication with the camera 120.

In the present disclosure, “processor” may refer to a computing device capable of performing image quality improvement and distortion compensation processing on one or more images photographed by the camera 120. In the present disclosure, “composite image” or “composite” The image "may refer to an image generated by synthesizing one or more images of the memo paper 190 accommodated in the paper holder 116 with the camera 120 having one or more focuses. In addition, "composite image" may refer to that distortion of the image is compensated to be the same or similar to the actual shape of the memo received in the paper holder 116.

The processor 150 may synthesize various methods in synthesizing one or more images photographed by the camera 120. For example, as described in detail below with reference to FIG. 9, one or more images may be synthesized by calculating an average value for each pixel of one or more images. Each pixel of the image may represent a value representing color (or color), grayness, or transparency, in which case each pixel of the synthesized image corresponds to one or more images in terms of color, grayness, or transparency. It has an average value of pixels.

Further, the processor 150 may perform distortion compensation of the synthesized image. As described above, when the camera 120 shoots at an angle inclined with respect to the upper plane of the paper holder 116, the shape of the note paper included in the captured image is compared with the shape of the actual note paper 190 When done, distortion may occur. Therefore, the processor 150 may compensate for distortion by converting the synthesized image in various ways in order to increase the readability of the contents of the memo included in the captured memo image. For example, the processor 150 may compensate for distortion of the synthesized image by applying a perspective projection transformation to the synthesized image. That is, as described in detail below with reference to FIG. 10, the processor 150 establishes between the projected corresponding points when the plane for the memo shape of the synthesized image is projected onto the plane for the actual memo shape. By calculating a constant transform relationship, distortion compensation can be performed.

The memo pad 110 may further include a storage unit 130 for storing an image or / and a composite image as illustrated in FIG. 3. The storage unit 130 may implement various storage methods when storing an image or / and a composite image, and may be linked to the processor 150 for this purpose.

For example, when the user drives the camera 120 through the input unit 140 to be described later, the processor 150 may store an image captured from the camera 120 to the storage unit 130. As another example, a motion detection sensor (not shown) is additionally provided in the memo pad 110, so that the user can tap the memo paper 190 a certain number of times or pass the memo paper 190 to identify the user. The motion detection sensor may be configured to detect an operation. In this configuration, when the motion detection sensor detects a user's motion, the processor 150 may drive the camera 120 to take an image and store the captured image in the storage 130. As another example, the memo paper 190 accommodated in the paper holder 116 through the image sensor 122 provided in the camera 120 or the image sensor provided separately in addition to the image sensor 122 provided in the camera 120 ) Can be configured to detect or measure image changes. In this configuration, when a change in the image photographed by the camera 120 is detected as the user progresses a memo on the memo paper 190, the processor 150 may store the image in the storage 130.

In the above-described examples, the storage unit 130 not only stores the image captured by the camera 120 with one or more focuses, but also a composite image generated by synthesizing one or more images by the processor 150, that is, a digital memo image. You can also save

The memo pad 110 may further include an input unit 140 that receives a user input driving the camera 120 and the processor 150 to generate a digital memo image as shown in FIG. 3. In one embodiment, the input unit 140 may be any one of a button, a touch sensor, and a motion sensor formed outside the memo pad 110, but is not limited thereto. In one example, the input unit 140 may receive the user's input and allow the camera 120 to photograph the memo paper 190 accommodated in the paper holder 116. In another example, the input unit 140 may generate a digital memo image by synthesizing one or more images photographed by the camera 120 by driving the processor 150 when receiving a user input driving the camera 120. Can be. In another example, the input unit 140 may receive a user input driving the processor 150 separately from a user input driving the camera 120 and allow the processor 150 to generate a digital memo image. .

As shown in FIG. 3, the memo pad 110 is a power supply unit that supplies power so that the camera 120, the storage unit 130, the input unit 140, the processor 150, and the communication unit 170 can be driven, respectively. It may further include (160). For example, the power supply unit 160 may be configured to receive power by wire or wireless from an external power supply. In another example, the power supply unit 160 may include a secondary battery capable of recharging power.

In addition, the memo pad 110 may further include a communication unit 170 that transmits a digital memo image to an external device, and the digital memo generating device 100 by the communication unit 170 is illustrated in FIGS. 5 and 6. As described above, various external devices may be connected through the network 250. Here, the network 250 is a communication network that is a high-speed period network of a large communication network capable of large-capacity, long-distance voice and data services, a wireless communication network for providing Internet or high-speed multimedia services, or Bluetooth, Wi-Fi ( It may be a local area wireless communication network including Wireless Fidelity, Beacon, and the like. In addition, the network 250 may include wired or wireless communication through micro-USB, USB, and the like.

As an example of an external device that can be connected to the digital memo generating device 100 and the network 250, a memo printer 210 may be included. The memo printer 210 may receive a digital memo image from the communication unit 170 of the digital memo generating device 100 and output it in the form of a memo paper.

As another example of the external device, a memo cloud service 220 may be included. The memo cloud service 220 receives and stores and manages the digital memo image from the communication unit 170 of the digital memo generating device 100, and the user can access the digital memo image or share it with other users when necessary. Can provide services. Here, the memo cloud service 220 may be configured to be interlocked with an existing note-taking application and to store or edit digital memo images in the note application. Here, the memo cloud service 220 may mean a server computer to which a user accesses to access and use information (data). The memo cloud service 220 may be operated by one or more server computers, a database (DB) including functional and structural combinations of software and hardware capable of storing and managing digital memo images, and information stored in the DB It may further include a DBMS (Database Management System) for searching, storing, and managing.

As another example of the external device, a terminal device such as a smart phone 230 and a portable computer 240 may be included. In this case, the user may receive a digital memo image from the digital memo generating device 100 to the terminal device and refer to or edit the content as necessary.

In addition, as shown in FIG. 6, in the digital memo storage system 300, the digital memo generating device 100 is connected to a terminal device such as a smart phone 230 or a portable computer 240 by wired or wireless communication. Can be connected. In this case, the digital memo generating device 100 may be connected to the network 250 by the

terminal devices

230 and 240 to interwork with the memo printer 210 or the memo cloud service 220.

7 is a view showing the configuration of the camera and the paper holder of the digital memo generating device according to another embodiment of the present disclosure, the components of the digital memo generating device 600, the digital memo generating device 100 shown in FIG. ). That is, the digital memo generating device 600 illustrated in FIG. 7 may include at least some of the functions or components of the digital memo generating device 100 illustrated in FIG. 1. In addition, among the functions or components of the digital memo generating device 600, for components having the same member number or name as described in FIG. 1 described above, a detailed description may be omitted to avoid repetition, , Can only explain changes or additional parts.

The digital memo generating device 600 may be provided such that the camera 120 is arranged on the lower portion 114 of the memo pad so that the focus of the camera 120 is directed upward. In this configuration, the light (or image) from the memo paper 190 accommodated in the paper holder 116 may be reflected by the top of the memo pad 112 by the mirror 610 and incident on the camera 120. Specifically , The image sensor 122 of the camera 120 has an imaging surface arranged substantially parallel to the upper plane of the paper holder 116, and the optical system of the camera 120 is light incident from the upper plane of the paper holder 116 It may include a mirror 610 arranged to be reflected on the imaging surface of the image sensor 122 by reflecting.

In this configuration, the focal length of the camera 120 can be increased so that the top (ie, the closest part from the camera 120) and bottom (ie, the most distant parts from the camera 120) of the memo paper 190 can be made. The difference in focal length can be made small. Accordingly, the camera 120 photographs the memo paper 190 arranged on the upper plane of the paper holder 116 with one or more focuses to generate and synthesize one or more images, thereby creating a digital memo image with improved readability. can do.

In the digital memo generating apparatus according to another embodiment of the present disclosure, the camera 120 is provided in the memo pad 110, but the processor 150 may be configured to be arranged independently from the memo pad 110. Alternatively, although the processor 150 is provided in the memo pad 110, the camera 730 may be configured to be arranged independently from the memo pad 110. In this case, the processor 150 may transmit and receive image data to and from the

cameras

120 and 730 through wireless or wired communication. In addition, the

cameras

120 and 730 may transmit one or more images of the upper plane of the paper holder 116 to the processor 150 so that the processor 150 can synthesize one or more images to generate a digital memo image. have.

8 is a view showing the configuration of a camera and a paper holder of a digital memo generating device according to another embodiment of the present disclosure. In this configuration, the processor 150 is provided in the memo pad 110, but the camera 730 is arranged independently from the memo pad 110. The components of the digital memo generating apparatus 700 may correspond to the components of the digital memo generating apparatus 100 shown in FIG. 1. That is, the digital memo generating device 700 illustrated in FIG. 8 may include at least a part of functions or components of the digital memo generating device 100 illustrated in FIG. 1. In addition, among the functions or components of the digital memo generating device 700, for components having the same member number or name as described in FIG. 1 described above, a detailed description may be omitted to avoid repetition. , Can only explain changes or additional parts.

The digital memo generating device 700 includes a configuration in which a smartphone 720 equipped with a camera 730 is arranged below the memo pad 110. In this configuration, the camera 730 of the smartphone 720 can be used to photograph the upper plane of the paper holder 116 with one or more focuses. Specifically, a hole through which the camera 730 is inserted may be formed in the lower portion 114 of the memo pad so that the focus of the camera 730 of the smartphone 720 faces upward, and the image sensor of the camera 730 is paper. A mirror arranged to have an imaging surface arranged parallel to the upper plane of the holder 116, and the optical system of the camera 730 to reflect light incident from the upper plane of the paper holder 116 and to be irradiated to the imaging surface of the image sensor ( 710). The mirror 710 may be provided on the memo pad top 112 to reflect the memo described in the memo paper 190 accommodated in the paper holder 116 with the camera 730.

9 is a flowchart illustrating a digital memo generation method according to an embodiment of the present disclosure. 10 is a diagram illustrating a procedure of synthesizing a plurality of memo images by a digital memo generation method according to an embodiment of the present disclosure. 11 is a diagram illustrating a focus area detection method according to an embodiment of the present disclosure. 12 is a diagram illustrating a method of synthesizing a plurality of memo images according to the size of a frequency component according to an embodiment of the present disclosure. 13 is a diagram illustrating a method of cropping and combining regions in focus in a plurality of images according to an embodiment of the present disclosure. In addition, FIG. 114 is a diagram illustrating a procedure for compensating for distortion of a memo image by a digital memo generation method according to an embodiment of the present disclosure.

The digital memo generation method 800 may be initiated by a step 810 in which the camera captures one or more memo images according to one or more focuses. In one embodiment, referring to FIGS. 1 to 8, the upper plane of the paper holder 116 accommodating the note paper 190 is arranged to shoot with at least one focus at an angle inclined to the upper plane By

cameras

120 and 730. One or more images of the upper plane of the paper holder 116 may be photographed.

At this time, the

cameras

120 and 730 are arranged to photograph the upper plane of the paper holder 116 with at least one focus at an angle inclined to the upper plane, and the upper plane of the paper holder 116 It may include an image sensor having an imaging surface arranged in parallel with. In addition, the light incident from the upper plane of the paper holder 116 may be reflected by the optical system of the cameras 120 and 730 (for example, the mirrors 610 and 710) and irradiated to the image sensor.

Next, the processor may synthesize one or more memo images photographed by the camera (830). In one embodiment, referring to FIGS. 1 to 8, the

cameras

120 and 730 take one or more memo images according to one or more focuses, and then by the processor 150, by the

cameras

120 and 730. One or more images taken can be synthesized. More specifically, when the processor 150 synthesizes one or more images photographed by the

cameras

120 and 730, for example, the one or more images may be synthesized by calculating an average value for each pixel of the one or more images. . As illustrated in FIG. 10, a composite image 940 may be generated by calculating an average value for each pixel of the first image, the second image, and the third image photographed at different focuses by the

cameras

120 and 730. have.

As another example, a focal region may be detected for each of one or more images photographed by the

cameras

120 and 730, and images having different focal regions may be calculated to synthesize an overall focused image. As illustrated in FIG. 11, a focus area may be detected by applying a hill climbing algorithm to each of the captured images. That is, it is possible to compare the reference values while searching for the contrast values from the furthest region to the closest region with the

cameras

120 and 730 on each image. At this time, the

areas

1110, 1120, and 1130 having a contrast value higher than the reference value and / or the area 1120 having the highest contrast value may be detected as a focus area.

Next, by calculating differences between a plurality of images having different focal regions, only high-frequency components can be extracted to synthesize a well-focused image as a whole. For example, as illustrated in FIG. 12, a difference image 1230 corresponding to a difference between the original image 1210 and a blurred image 1220 is obtained. The difference image 1230 represents the high frequency component of the original image 1210. If the difference image 1230 is multiplied by an appropriate constant value (α) and added to the blur image, the difference image 1230 has the same sharpness or the original image 1210. An image with a higher definition than 1210 can be obtained.

As another example, a focal region may be detected for each of the one or more images photographed by the

cameras

120 and 730, and a fragmented image corresponding to each focal region may be cropped to synthesize an overall focused image. have. Referring back to FIG. 10, an image 940 may be generated by synthesizing the focal region 910 of the first image, the focal region 920 of the second image, and the focal region 930 of the third image, respectively. . As illustrated in FIG. 13,

contrast values

1340, 1350, and 1360 on three images photographed with different focuses are compared with a reference value, and

areas

1310, 1320 that have areas having a contrast value higher than or equal to the reference value are in focus. , 1330). Here, the region 1370 is an overlapping section where the focal regions overlap. Focus areas that are not overlapping areas are used as they are, and overlap areas are synthesized by calculating the average value for each pixel of two images, or by using the contrast value as a weight to multiply and sum the pixel values of each image. Can be.

Next, the processor may perform distortion compensation of the synthesized image (830). In one embodiment, when the processor 150 performs distortion compensation of a synthesized image, distortion compensation of the synthesized image may be performed by applying a perspective projection transformation to the synthesized image.

As a specific example, as illustrated in FIG. 10, the focus areas 910 of the first image, the focus areas 920 of the second image, and the focus of the third image taken by the

cameras

120 and 730 with different focuses An image 940 that is a composite of the region 930 may be generated, and distortion compensation of the synthesized image 940 may be performed to compensate for distortion to be the same / similar to the shape of the memo described on the note paper 190 by the user. An image 950, that is, a digital memo image may be generated. In converting the synthesized image 940 into a distortion-compensated image 950, as shown in FIG. 10, four vertices (x1, y1), (x2, in the shape of a memo included in the image 1010 before distortion compensation) y2), (x3, y3), (x4, y4) each of the four vertices (X1, Y1), (X2, Y2), (X3, Y3) of the memo shape included in the image 1020 after distortion compensation, Perspective projection transformation may be applied to transform the distorted image by projecting for each of (X4, Y4).

Next, the communication unit may transmit the distortion-compensated image to an external device or a memo cloud service (840). In one embodiment, referring to FIGS. 1 to 7, after generating a digital memo image, the digital memo image may be transmitted to an external device by the communication unit 170. The external device may include a terminal connected through a wired / wireless communication network such as a memo printer 210, a memo cloud service 220, a smart phone 230, or a portable computer 240, as described above.

11 is a flowchart illustrating a method for generating and storing a digital memo according to another embodiment of the present disclosure. The steps of the digital memo generation and storage method 1500 may correspond to at least some of the steps of the digital memo generation method 800 shown in FIG. 9. Specifically, steps 1510, 1520, and 1530 of the digital memo generating and storing method 1500 shown in FIG. 11 are

steps

810, 820, and 830 of the digital memo generating method 800 shown in FIG. ). Therefore, hereinafter, for convenience of explanation, in order to avoid repetition, detailed description of

steps

1510, 1520, and 1530 is omitted, and only changes or additional parts are described.

According to one embodiment, in step 1510, a camera arranged to photograph the upper plane of the paper holder 116 for receiving the note paper 190 with at least one focus at an angle inclined to the upper plane ( 120, 730). One or more images of the upper plane of the paper holder 116 may be photographed. Next, in step 1520, after the processor 150 synthesizes one or more memo images, distortion compensation of the image synthesized in step 1530 may be performed.

After generating the digital memo image as described above, the processor 150 may determine whether to store the digital memo image, and may store the digital memo image in a storage device inside or outside the digital memo generation device as necessary.

Specifically, in step 1540, the processor determines whether the memo writing is completed. In one embodiment, referring to FIGS. 1 to 8, the processor 150 may determine whether or not the memo writing is completed through various sensors or user input. If the processor 150 determines that the memo writing is not completed, the controller 150 may return to step 1510 and control the

cameras

120 and 730 to shoot a new image. On the other hand, when the processor 150 determines that the memo is completed, the processor 150 determines whether the generated digital memo image has already been stored.

In step 1550, the processor determines whether the digital memo image has already been stored. In one embodiment, referring to FIGS. 1 to 8, the processor 150 stores the generated digital memo image in contrast to the digital memo image stored in the storage 130 of the digital memo generating device 100. It can be determined whether or not. In another embodiment, the processor 150 determines whether the generated digital memo image is a memo already stored in contrast to the digital memo image stored in the memo printer 210, the memo cloud service 220, and the

portable computing devices

230 and 240. It can also be determined. If it is determined that the digital memo image has already been stored, the camera 120 may return to step 1510 and control the camera 120 to shoot a new image. On the other hand, if it is determined that the digital memo image is not already stored, the processor 150 stores the generated digital memo image in the storage unit 130 or an external device (1560).

The digital memo generating apparatus and method according to various embodiments of the present disclosure described above may increase the management and utilization of the memo by generating and storing a digital memo image by photographing a memo written by the user with a camera. , The digital memo image generated as described above can be conveniently output to a memo printer or transmitted to another computer or a memo cloud service system to conveniently manage and recycle the digital memo image.

Various embodiments of the present disclosure may be embodied as computer readable codes on a computer readable recording medium. That is, as a non-transitory computer-readable storage medium in which instructions for digital memo generation are stored, when the instructions are executed by a processor, the processor causes the upper plane of the paper holder 116 to receive the note paper 190. Photographing one or more images of the upper plane of the paper holder 116 by

cameras

120 and 730 arranged to shoot with at least one focus at an angle inclined to the upper plane; And a computer readable storage medium that performs an operation of synthesizing one or more images captured by the

cameras

120 and 730 to generate a digital memo image.

Computer-readable storage media include all types of recording devices in which data that can be read by a computer system are stored, examples of which include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. In addition, the computer readable storage medium may be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. And functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers in the technical field to which the present invention pertains.

The preceding description of the present disclosure is provided to enable those skilled in the art to make or use the present disclosure. Various modifications of the present disclosure will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to various modifications without departing from the spirit or scope of the present disclosure. Accordingly, the present disclosure is not intended to be limited to the examples described herein, but is intended to be given the broadest scope consistent with the principles and novel features disclosed herein.

Claims

As a digital memo generating device,

A paper holder that can accommodate note paper;

A camera including an optical system and an image sensor arranged to photograph the upper plane of the paper holder with at least one focus at an angle inclined to the upper plane; And

And a processor that generates a digital memo image by synthesizing one or more images taken by the camera.
According to claim 1,

The optical system of the camera is arranged such that the optical axis of the optical system has an angle of less than 90 degrees with respect to the upper plane of the paper holder,

Digital memo generation device.
According to claim 1,

The optical system of the camera includes a lens configured to have at least one focus with respect to the upper plane of the paper holder,

Digital memo generation device.
According to claim 1,

The image sensor of the camera has an imaging surface arranged parallel to the upper plane of the paper holder,

The optical system of the camera includes a mirror arranged to reflect light incident from the upper plane of the paper holder to be irradiated to the imaging surface of the image sensor,

Digital memo generation device.
According to claim 1,

The processor synthesizes the one or more images by synthesizing a focal region of the one or more images,

Digital memo generation device.
According to claim 1,

The processor performs distortion compensation of the synthesized image by applying a perspective projection transformation to the synthesized image,

Digital memo generation device.
According to claim 1,

Further comprising a communication unit for transmitting the digital memo image to an external device,

Digital memo generation device.
According to claim 1,

Further comprising an input unit for receiving a user input for driving the camera and the processor to generate the digital memo image,

Digital memo generation device.
As a digital memo generating device,

A paper holder that can accommodate note paper; And

A camera including an optical system and an image sensor arranged to photograph the upper plane of the paper holder with at least one focus at an angle inclined to the upper plane,

The camera is configured to transmit one or more images of the upper plane of the paper holder to a processor, and the processor is configured to synthesize the one or more images to generate a digital memo image.
The method of claim 9,

The optical system of the camera is arranged such that the optical axis of the optical system has an angle of less than 90 degrees with respect to the upper plane of the paper holder,

Digital memo generation device.
The method of claim 9,

The optical system of the camera includes a lens configured to have at least one focus with respect to the upper plane of the paper holder,

Digital memo generation device.
The method of claim 11,

The image sensor of the camera has an imaging surface arranged parallel to the upper plane of the paper holder,

The optical system of the camera includes a mirror arranged to reflect light incident from the upper plane of the holder to be irradiated to the imaging surface of the image sensor,

Digital memo generation device.
The method of claim 9,

Further comprising a communication unit for transmitting the image to the processor,

Digital memo generation device.
The method of claim 9,

Further comprising an input unit for receiving a user input to drive the camera to shoot the image,

Digital memo generation device.
As a method for generating a digital memo,

Photographing one or more images of the upper plane of the paper holder by a camera arranged to photograph the upper plane of the paper holder accommodating the note paper with at least one focus at an angle inclined to the upper plane;

And generating, by a processor, a digital memo image by synthesizing the one or more images captured by the camera.
The method of claim 15,

The camera includes an optical system arranged to photograph the upper plane of the paper holder with at least one focus at an angle inclined with respect to the upper plane, and an image having an imaging plane arranged parallel to the upper plane of the paper holder Includes a sensor,

The method further comprises, by the optical system of the camera, reflecting light incident from the upper plane of the paper holder and irradiating the image sensor.

How to create a digital note.
The method of claim 15,

The step of generating the digital memo image includes synthesizing the one or more images by synthesizing a focal region of the one or more images by the processor.

How to create a digital note.
The method of claim 15,

The generating of the digital memo image may include, by the processor, applying distortion correction of the synthesized image by applying a perspective projection transformation to the synthesized image,

How to create a digital note.
The method of claim 15,

Further comprising, by the communication unit, transmitting the synthesized image to an external device,

How to create a digital note.
A non-transitory computer-readable storage medium on which instructions for digital memo generation are stored, which when executed by a processor causes the processor to:

Photographing one or more images of the upper plane of the paper holder by a camera arranged to photograph the upper plane of the paper holder accommodating the note paper with at least one focus at an angle inclined to the upper plane; And

A computer readable storage medium for performing an operation of synthesizing one or more images taken by the camera to generate a digital memo image.