JP2020119051A - Handwritten memo extraction program - Google Patents

Abstract

To provide a handwritten memo extraction program that can take advantage of both a merit of schedule management in electronic data and a merit of schedule management by handwriting on a pocket notebook or a calendar.SOLUTION: A handwritten memo extraction program includes the steps of: obtaining an image of a calendar on which a user's handwritten memo is written; cutting out an image of an area where the handwritten memo is written from the image of the calendar; and saving the image of the area where the handwritten memo is written while associating the date corresponding to the area therewith. Especially, the handwritten memo extraction program further includes the steps of: setting a URL for accessing the image of the area where the handwritten memo is written via the Internet; and sending the URL and the corresponding date to a schedule management server that manages a user's schedule and updating the user's schedule.SELECTED DRAWING: Figure 4

Description

The present invention relates to a handwritten memo extraction program that can be easily used together by taking advantage of the advantages of both schedule management with electronic data and schedule management by handwriting on a notebook or calendar.

The penetration rate of smartphones exceeds 80%, and more and more people are managing schedules using smartphones. It can be said that a smartphone has excellent portability and immediacy, and that schedule management using the smartphone is highly convenient because of searchability, ease of storage, and the like (see Patent Document 1).

There are also people who manage schedules using a personal computer. Although schedule management on a personal computer is not as portable or instant as a smartphone, complicated input and editing are also easy. Furthermore, some people use a tablet located in the middle of them to perform schedule management.

On the other hand, due to reliance on electronic data that is easy to modify and delete, and a sense of anxiety about malfunctions of smartphones and PCs, and the ease of handwriting and ease of use, conventional paper notebooks and calendars were used. There are still many people who write down their plans.

Japanese Patent Laid-Open No. 2018-041289

Therefore, there are merits and demerits in the schedule management by electronic data and the schedule management by handwriting on a notebook or calendar. In fact, even a person who manages a schedule with electronic data may also use a schedule management by handwriting on a conventional paper notebook or calendar.

However, if schedule management is performed using electronic data and paper, it is necessary to write the schedule in both, which is very troublesome. Further, if the data is written in only one side, the schedule may be overlooked.

Therefore, an object of the present invention is to provide a handwritten memo extraction program capable of easily shifting writing by handwriting on a notebook or calendar to schedule management by electronic data.

In order to solve the above problems, a program for extracting a handwritten memo according to one aspect of the present invention includes a step of acquiring an image of a calendar in which a user's handwritten memo is written, and a handwritten memo written from the image of the calendar. A step of cutting out the image of the area, a step of storing the image of the area in which the handwritten memo is written in association with the date corresponding to the area, and a step of saving the image of the area in which the handwritten memo is written via the Internet Setting a URL for access, transmitting a date corresponding to the URL to a schedule management server that manages the schedule of the user, updating the schedule of the user, the URL and the date And a step of creating an iCalendar format ICS file including the following, and transmitting the ICS file to the schedule management server, thereby transmitting the date corresponding to the URL.

Further, in one embodiment, the calendar displays dates for one month, and the year and month are specified from the positions of the days of the first and last days of the month in the image of the calendar.

According to the handwritten memo extraction program of the present invention, schedule management by handwriting on a notebook or calendar and schedule management by electronic data can be easily used together.

FIG. 1 is a network diagram for explaining an embodiment of a handwritten memo management service by a handwritten memo management server according to the present invention. FIG. 2 is a diagram showing a specific example of a calendar in which a schedule or a memo is written by hand, which is handled by the handwritten memo extraction program according to the first embodiment of the present invention. FIG. 3 shows an example of an image of the handwritten memo area extracted by the handwritten memo extraction program according to the first embodiment of the present invention. FIG. 4 is a flowchart illustrating the operation of the handwritten memo extraction program according to the first embodiment of the present invention. FIG. 5 is a diagram showing a state in which the user data of the schedule management server updated by the handwriting memo extraction program according to the first embodiment of the present invention is opened by the calendar application. FIG. 6 is a diagram showing a specific example of a calendar in which a schedule and memos are written by hand, which is handled by the handwritten memo extraction program according to the second embodiment of the present invention. FIG. 7 shows an example of an image of a handwritten memo area extracted by the handwritten memo extraction program according to the second embodiment of the present invention. FIG. 8 shows an image of another handwritten memo area extracted by the handwritten memo extraction program according to the second embodiment of the present invention.

An embodiment of a handwritten memo extraction program according to the present invention will be described below with reference to the accompanying drawings. This handwritten memo extraction program is installed in a handwritten memo management server that connects to the Internet and provides a user with a handwritten memo management service.

FIG. 1 is a network diagram for explaining an embodiment of a handwritten memo management service by the handwritten memo management server 1. Here, in addition to the handwritten memo management server 1, the schedule management server 3, the smartphone 7 carried by the user 5, and the personal computer 9 used by the user 5 in the office are connected to the Internet.

The user 5 has an account in both the handwritten memo management server 1 and the schedule management server 3, and can use the handwritten memo management service and the schedule management service. The schedule management server 3 synchronizes and manages the schedules and memos input by the user 5 via the calendar application installed on the smartphone 7 and the personal computer 9. The user 5 also registers account information such as a user ID and a password when using the schedule management server 3 in the handwritten memo management server 1. With this, the handwritten memo management server 1 can access the data stored and managed by the schedule management server 3.

User 5 writes the schedule and memos on the desk calendar. FIG. 2 is a diagram showing a specific example of a calendar in which an appointment and a memo are written by hand. This calendar is a calendar dedicated to handwritten memo management, in which the position detection symbol S is printed at the lower left.

In order to use the handwritten memo management service, the user 5 shoots, for example, the entire month of the calendar (FIG. 2) with the smartphone and sends it to the handwritten memo management server 1. The handwritten memo management server 1 analyzes the received shooting data, cuts out each handwritten memo as an image, and stores it in association with the date.

That is, the handwriting memo management server 1 character-recognizes the year and month display at the upper right of the image (here, “January 2019”), further recognizes the position detection symbol S printed at the lower left, and Identify the location of each date area in between. Then, it is confirmed whether or not each region exists in the handwritten memo, and for the existing region, an image of the region (see FIG. 3) is cut out and stored in association with the date.

In the example of FIG. 2, images of the corresponding areas are displayed on January 7, 2019, January 9, 2019, January 11, 2019, January 21, 2019, and January 24, 2019, respectively. It is associated and saved. Although "Adult's Day" is also printed on January 14, 2019, holidays are registered in advance in the handwritten memo extraction program, and it is recognized that the memo is not a handwritten memo.

A URL is set for each image data so that it can be accessed as a resource on the Internet. For example, a URL such as "http://www.tegakimemo.co.jp/user5/20190107.jpg" is set. Then, an iCalendar ICS file (for example, user520190107.ics) is also created. The URL is described in the event of this file.

For example, "handwritten memo" is described as the event title (SUMMARY), and the URL is described as the content (DESCRIPTION). Also, the date (for example, January 7, 2019) is described as the date (DTSTART and DTEND), and the creation date (shooting date) of this image data is described as the creation date (CREATED). This ICS file enables cooperation with general calendar software.

Then, the handwritten memo management server 1 accesses the schedule management server 3, transmits the ICS file, and updates the data of the user 5 with these handwritten memos. For example, "Handwritten memo" is entered as the title and the above URL, that is, "http://www.tegakimemo.co.jp/user5/20190107.jpg" is entered as the content in the schedule of January 5, 2019 of user 5. It Other handwritten notes are also entered into the schedule for the corresponding dates.

FIG. 4 is a flow chart for explaining the operation of the handwritten memo extraction program. First, in step S1, the handwritten memo management server 1 acquires a calendar image taken by the smartphone 7. In step S2, the captured calendar image is analyzed to specify the position of each date area. In step S3, n is set to 1, and in step S4, it is checked whether or not a handwritten memo exists in the nth area (first date area).

If the handwritten memo exists in step S4, the analysis of the n-th area including the handwritten memo is cut out and stored in association with the corresponding date in step S5. In step S6, it is confirmed whether or not all the areas have been checked. If all areas are not checked, n is incremented in step S7 and the process returns to step S4. If all the areas are checked, the image of the area including the handwritten memo saved in step S8 is input to the schedule management server 3.

As a result, the handwritten memo written in the calendar is input to the schedule management server 3. When the user 5 accesses the schedule management server 3 with the smartphone 7 or the personal computer 9 and opens the schedule of January 7, 2019 with a calendar application, for example, the above URL is displayed (see FIG. 5) and the URL is displayed. When clicked, a handwritten memo as shown in FIG. 3 is displayed. Also, by using this calendar application, a reminder, a Todo list, a notification function, and the like can be easily added to a handwritten memo schedule.

In the above example, the calendar is displayed for one month, but a calendar in which two months are displayed together can be processed in the same manner. In this case, two months worth of images are taken as one image, and two months worth of handwritten notes are cut out for each date and extracted. The same applies to a calendar in which three months or more are displayed together.

Further, in the case of a personal computer, since the screen of the calendar application can be widened, the image of the handwritten memo may be directly displayed in each date column of the display for one month. In this case, the display screen of the calendar application of the personal computer has an appearance similar to that shown in FIG. 2, but since it is electronic data on the screen, it is possible to add a schedule by clicking each date.

Furthermore, if the ICS file is created as described above from the extracted handwritten memo image and then this ICS file is edited, a Todo list composed of a part of the handwritten memo included therein can be created. By setting the notification date and time in the handwritten memo of this Todo list, you can easily set reminders by linking with existing applications.

Normally, in the service of the schedule management server 3, when inputting a memo through a calendar application, it is possible to specify a time together with a date. In the first embodiment, since the handwritten memo is cut out for each date and extracted from the calendar for one month, the time is not designated. However, as shown in FIG. 6, there is also a notebook in which writing can be performed every time. In this notebook, one week's worth is divided into hours. In the second embodiment, such a notebook is photographed and a handwritten memo is extracted.

In this case, the smartphone 7 also captures an image for a week, which is a timetable, divides the image for each time slot, and stores the image of the handwritten memo in association with each date and time. In the example of FIG. 6, the smartphone 7 takes a picture of the notebook that opened the page for the second week of January 2019. This notebook has a predetermined layout for handwritten memo management, and the position detection symbol S is also printed at the lower left.

The captured image is transmitted to the handwritten memo management server 1. The handwritten memo management server 1 analyzes the received shooting data, cuts out each handwritten memo as an image, and stores it in association with the date and time.

That is, the handwriting memo management server 1 recognizes the year display (here, “2019”) on the upper right of the image, further recognizes the position detecting symbol S printed on the lower left, and identifies each date between them. Specify the position in the time domain of. Ruled lines are also used to identify the position. Then, it is determined for each area whether or not a handwritten memo exists, and for the existing area, an image of the area (see FIG. 7) is cut out and stored in association with the date and time.

In the example of FIG. 6, the images of the corresponding areas are stored in association with 17:00 on January 7, 2019, 10:00 on January 9, 2019, and 18:00 on January 11, 2019, respectively. Here again, the URL is set for each image data so that it can be accessed as a resource on the Internet. For example, a URL such as "http://www.tegakimemo.co.jp/user5/2019010711.jpg" is set for an image associated at 11:00 on January 7, 2019.

Then, similarly to the first embodiment, an iCalendar format ICS file (for example, user52019010711.ics) is also created. The URL is described in the event of this file.

For example, “handwritten memo” is described as the event title (SUMMARY), and the above URL is described as the content (DESCRIPTION). The date and time (for example, 17:00 on January 7, 2019) is described as the date (DTSTART and DTEND), and the creation date (shooting date) of this image data is described as the creation date (CREATED). After all, this ICS file enables linkage with general calendar software (google calendar, etc.).

Then, the handwritten memo management server 1 accesses the schedule management server 3 and updates the data of the user 5 with this ICS file. For example, in the schedule of the user 5 at 11:00 on January 7, 2019, enter "handwritten memo" as the title and "http://www.tegakimemo.co.jp/user5/2019010711.jpg" as the content. Other handwritten memos are similarly input to the schedule of the corresponding date and time.

As a result, the handwritten memo written in the calendar is input to the schedule management server 3. When the user 5 accesses the schedule management server 3 with the smartphone 7 or the personal computer 9 and opens the schedule at 11:00 on January 7, 2019, the above URL is displayed as the content. A handwritten memo as shown is displayed.

In the example of FIG. 6, the leader line is also written after 18:00 on January 11, 2019, 19:00 on January 11, 2019 and 20:00 on January 11, 2019. In this way, when there is continuous writing over a plurality of time zones, as shown in FIG. 8, an image including the plurality of time zones is cut out. A URL such as "http://www.tegakimemo.co.jp/user5/2019011118-20.jpg" is set for this image.

In this case, the created ICS file describes “handwritten memo” as the title of the event (SUMMARY) and the above URL as the content (DESCRIPTION), but the start time (DTSTART) ends on January 11, 2019 at 18:00. As the time (DTEND), 21:00 on January 11, 2019 is described. Also, the creation date (shooting date) of this image data is described as the creation date (CREATED).

As described above, according to the handwritten memo extraction program of the present invention, the advantages of both schedule management by electronic data and schedule management by handwriting on a notebook or calendar can be utilized, and they can be easily used together. ..

In the above embodiment, a dedicated calendar having a predetermined layout for the handwritten memo extraction program is used, but the present invention is not limited to this method. The handwritten memo extraction program can be implemented by using a general calendar in which an array of dates (and times) is printed. In the case of a general calendar, image processing becomes slightly complicated, but it is sufficient to identify the area of each date by recognizing numbers and determining the date position.

For example, even a general calendar displaying one month is common in that the dates are arranged in the rectangular area in the order of the day of the week. The arrangement of dates in this rectangular area depends on the year and month. For example, in January 2019, one day starts from the position of Tuesday and the last day is Thursday 31st. The handwritten memo extraction program can identify that the month having such an arrangement near the current time is January 2019 by collating with the internal calendar.

Therefore, even if the year and month cannot be obtained with a general calendar, the handwritten memo extraction program uses the picked-up image of the month display to position the days of the first and last days of the month, that is, the 1st and 31st ( Or, specify the year and month from the position of the day of the week (30th).

In the above embodiment, one calendar is photographed and the handwritten memo is input to the schedule management server 3, but the present invention is not limited to this method. That is, even when the user uses two or more calendars together, if they are photographed and transmitted to the handwritten memo management server 1, the handwritten memo management server 1 also analyzes the respective photographed data received. Then, the handwritten memo of each calendar is cut out as an image, and both are saved as the data of the user in association with the date (and time). If there are handwritten memos on the same date in multiple calendars, for example, refer to each handwritten memo image "http://www.tegakimemo.co.jp/user5/20190107a.jpg", "http://www Set multiple URLs such as ".tegakimemo.co.jp/user5/20190107b.jpg" for that date. As the event content of the ICS file, the plurality of URLs are described with a delimiter such as a line feed interposed therebetween. As described above, handwritten memos of a plurality of calendars used by the same user can be put together into one schedule management service and can be uniformly referred to by corresponding calendar applications, which is highly useful.

Further, as described above, it is possible that the calendar is photographed and the handwritten memo is extracted, and then additional writing is performed on the same calendar. In such a case, the same calendar is taken again to extract the handwritten memo. In that case, multiple handwritten memo images may be associated with the same date, but in this case, for example, refer to each handwritten memo image "http://www.tegakimemo.co.jp/user5". Set multiple URLs such as "/20190107.jpg" and "http://www.tegakimemo.co.jp/user5/20190107-1.jpg" to that date. After all, as the event contents of the ICS file, the plurality of URLs are described with a delimiter such as a line feed interposed therebetween. By giving such a file name, the temporal order of each image can be recognized by the number after the hyphen.

The image taken by the smartphone 7 is usually in the exif format and includes the position information. It is also possible to identify the photographed calendar by using this position information. For example, since the position information is different between the home calendar and the work calendar, it is possible to determine from the position information whether the image of the handwritten memo extracted from that is the data of the other calendar. In this case, the title is, for example, "handwritten memo (office)" or "handwritten memo (home)" so that it can be known which data the handwritten memo is. If there are multiple handwritten memo images on the same date, for example, by using the title "Handwritten memo (office/home)", the first image is from the office calendar and the next image is from your home. You can see that it belongs to the calendar.

In this case, the above URL is "http://www.tegakimemo.co.jp/user5/calendar01/20190107.jpg", "http://www.tegakimemo.co.jp/user5/calendar02/20190107.jpg" For example, separate directories for each calendar and save.

A function for acquiring the position information of the handwritten memo image may be added to the calendar application, and the notification (alarm) may be set using the position information. That is, on the notification setting screen of the calendar application, the most recent shooting location (2 to 3 locations) is displayed, this shooting location is selected as appropriate, and when the selected shooting location is reached, the corresponding handwritten memo image is reminded. May be displayed as.

For example, there is a case where the next meeting schedule is written on the calendar of the notebook after the meeting at the place where the person is out. At this time, add a note about reminding you at the next meeting (contents like memorandum), and then shoot the calendar. Then, a notification (alarm) based on the position information of the handwritten memo image of the day is set. That is, at the time of the next meeting, when the user arrives at the place, the added memorandum content is notified and reminded.

There are also usage methods such as using the remind function in combination with the calendar at home or at work. For example, it is possible to give a notification when the user is at a specific date and time (for example, 1 pm to 2 pm).

1 Memo management server 3 Schedule management server 5 User 7 Smartphone 9 PC S Symbol for position detection

Claims (2)

A step of obtaining an image of the calendar in which the user's handwritten notes are written,
From the image of the calendar, a step of cutting out the image of the area where the handwritten memo is written,
A step of storing an image of the area in which the handwritten memo is written in association with a date corresponding to the area;
A step of setting a URL for accessing the image of the area where the handwritten memo is written via the Internet;
Sending a date corresponding to the URL to a schedule management server that manages the schedule of the user, and updating the schedule of the user,
Creating an iCalendar ICS file containing the URL and the date;
To the computer,
A handwriting memo extracting program, wherein the date corresponding to the URL is transmitted by transmitting the ICS file to the schedule management server. 2. The calendar displays dates for one month, and the year and month are specified from the positions of the days of the first and last dates of the month in the image of the calendar. Handwritten memo extraction program.

Images