KR20210045723A - Method for ocr based refrigerator food information automatic storage by taking receipt - Google Patents

Abstract

According to the present invention, the existing optical character recognition (OCR) technology has a lower recognition rate of Korean characters than other characters, and the recognition rate is different depending on the font. So, it is possible to develop and provide a technology to solve this problem. In addition, unlike other applications adopting the repeated input method, through one receipt recognition, user convenience can be provided by automation and weight reduction of input, and the expiration date that had to be entered individually for each product is digitalized to provide automatically the average expiration date and user convenience. A method for OCR based refrigerator food information automatic storage may include the steps of: receiving an image of a receipt; generating a first list; generating a second list; and retrieving and extracting a diet.

Description

OCR-based refrigerator food information automatic storage method through receipt photographing {METHOD FOR OCR BASED REFRIGERATOR FOOD INFORMATION AUTOMATIC STORAGE BY TAKING RECEIPT}

The present invention relates to a method for automatically storing food information in a refrigerator based on OCR through photographing a receipt, and more particularly, when purchasing food, photographing the receipt, recognizing it based on OCR, listing the product, and It relates to the management of the expiration date and dietary management.

Based on data from the Ministry of Environment in 2016, the average food waste disposal cost per year costs about 1 trillion won per year, and when the food waste is converted into the value of food resources, it is said to be over 20 trillion won. Among them, foods that have passed the expiration date are discarded about 600 billion won per year. In addition, the UK research institute'Waste and Resources Action Program (WRAP)' said in a report prepared based on data from the Food and Agriculture Organization of the United Nations (FAO) that it costs about 439 trillion dollars worldwide. In addition, according to the Korea Institute for Industrial Relations, a 15.7% reduction in food waste will reduce the annual treatment cost of 1268 billion won in Korea, and it is predicted that saving food resources will result in an economic benefit of 4 trillion won.

In addition, health care in a broad sense means an overall health care business that combines the concepts of disease prevention and management from existing medical services in the treatment sector. The marketability of healthcare-related products has continued to increase since 2014, and the growth rate of 2020 is expected to be 4.8 times compared to the 2014 year. Accordingly, the marketability is sufficient and the trend continues to increase even now.

Accordingly, there is a service that manages purchased food, etc., but a method of repeatedly inputting a barcode or product of a product, not a receipt, is adopted, which causes inconvenience in use.

In addition, since there is no content other than the service for the expiration date notification, diversity is reduced, and there are restrictions on obtaining consumer's dietary information for providing customized diet management.

The present invention is to solve the problems of the prior art described above, by using OCR technology, by photographing a receipt, various information can be automatically input, through which, a user-customized recipe is recommended, and through the platform. Its purpose is to enable communication with users.

In addition, by providing data to experts, it may be possible to provide a service for providing and managing a recommended diet.

As a technical means for achieving the above-described technical problem, the OCR-based refrigerator food information automatic storage method through receipt photographing according to an embodiment of the present invention is a Create a product list by acquiring product information through OCR technology from the photographed and acquired image, create a list including the expiration date for the products in the list, and use the expiration date and the products in the list You can provide a recipe or diet.

In the present invention, since the conventional optical character recognition (OCR) technology has a lower Hangul character recognition rate than other characters, and the recognition rate is different depending on the font, it is possible to develop and provide a technology capable of solving this.

Unlike other applications adopting a method of repeatedly entering one by one, it is possible to provide user convenience by automating input and reducing weight through a single receipt recognition.

It is possible to provide user convenience by automatically providing the average expiration date by converting the expiration date, which had to be entered individually for each product, into data.

A different recipe may be provided according to the user's preferred food based on the data information on the received product and the user's preference information, not indiscriminately providing a recipe.

You can receive professional diet management by experts similar to conventional diet recommendation programs, but unlike conventional diet recommendation programs, it is possible to check the user's purchased products and preferences, so efficient diet management based on the user's preferences and purchased products Service can be provided.

1 is an exemplary diagram of a system for performing an OCR-based automatic storage method of food information in a refrigerator through photographing a receipt according to an embodiment of the present invention.
2 is a flowchart of a conventional expiration date notification service.
3 is a flowchart illustrating a method of automatically storing food information in a refrigerator based on OCR through photographing a receipt according to an embodiment of the present invention.
4 is a flowchart illustrating a data processing process of a method for automatically storing food information in a refrigerator based on OCR through photographing a receipt according to an embodiment of the present invention.
5 is a flowchart illustrating a method of automatically storing food information in a refrigerator based on OCR through photographing a receipt according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected" with another part, this includes not only "directly connected" but also "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary.

In the present specification, the term "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Further, one unit may be realized by using two or more hardware, or two or more units may be realized by one piece of hardware. Meanwhile,'~ unit' is not meant to be limited to software or hardware, and'~ unit' may be configured to be in an addressable storage medium or configured to reproduce one or more processors. Thus, as an example,'~ unit' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, and procedures. , Subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays and variables. Components and functions provided in the'~ units' may be combined into a smaller number of elements and'~ units', or may be further separated into additional elements and'~ units'. In addition, components and'~ units' may be implemented to play one or more CPUs in a device or a security multimedia card.

The "terminal" mentioned below may be implemented as a computer or portable terminal that can access a server or other terminal through a network. Here, the computer is, for example, a notebook equipped with a web browser, a desktop, a laptop, a VR HMD (for example, HTC VIVE, Oculus Rift, GearVR, DayDream, PSVR, etc.). It may include. Here, VR HMD is for PC (for example, HTC VIVE, Oculus Rift, FOVE, Deepon, etc.), for mobile (for example, GearVR, DayDream, Storm Horse Racing, Google Cardboard, etc.) and console (PSVR) and It includes all stand alone models that are implemented independently (eg, Deepon, PICO, etc.). Portable terminals are, for example, wireless communication devices that guarantee portability and mobility, as well as smart phones, tablet PCs, and wearable devices, as well as Bluetooth (BLE, Bluetooth Low Energy), NFC, RFID, and ultrasound. , Infrared, Wi-Fi, Li-Fi may include various devices equipped with a communication module such as. In addition, "network" refers to a connection structure in which information exchange is possible between nodes such as terminals and servers, and includes a local area network (LAN), a wide area network (WAN), and the Internet. (WWW: World Wide Web), wired and wireless data networks, telephone networks, wired and wireless television networks, etc. Examples of wireless data communication networks include 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, Bluetooth communication, infrared communication, and ultrasound. Communication, Visible Light Communication (VLC), LiFi, and the like are included, but are not limited thereto.

Hereinafter, a system for performing an OCR-based refrigerator food information automatic storage method through receipt photographing according to an embodiment of the present invention will be described in detail with reference to FIG. 1.

According to an embodiment of the present invention, the system 10 for performing an OCR-based refrigerator food information automatic storage method through receipt photographing may include a server 100 and a user terminal 200, and may include a server ( 100) and the user terminal 200 may be connected through a wired/wireless network.

The server 100 according to an embodiment of the present invention may store information collected from the user terminal 200 and build a database.

In this case, the database information may be information on expiration date, statistics on purchased products, user's food preference, and customized diet management.

In addition, the average shelf life of each of the plurality of products and a recipe that can be cooked with a combination of each product may be stored as big data.

According to an embodiment of the present invention, the server 100 may collect user information, classify it into a database suitable for each service, and manage it.

In addition, it may be possible to collect meta data related to a user from big data, and classify and manage it according to the type and use or not according to a preset criterion.

It may be possible to collect and manage recommended information by collecting metadata and user information through an algorithm of the server.

Here, the server's algorithm may be an algorithm that collects and classifies information necessary for providing personalized services for each user, and extracts and classifies the information so that it can be used for services. I can.

According to an embodiment of the present invention, the user terminal 200 may include a camera module, and may include a memory in which an OCR-based refrigerator food information automatic storage program through receipt photographing is stored, and a processor that executes the program. have.

In addition, the user terminal 200 may be a smartphone, a tablet PC, a PC, and the like, and may be an electronic device in which an OCR-based refrigerator food information automatic storage application through receipt photographing is pre-stored.

The processor of the user terminal 200 may receive an image of a receipt from the camera module of the user terminal according to the execution of the program, and apply OCR technology to this, to create a list of product names in the image, It is possible to search and extract the expiration date corresponding to each product name from the database of the server 100, and search and extract a recipe corresponding to each product name in consideration of user preference.

Hereinafter, a service flow of an OCR-based refrigerator food information automatic storage method through receipt photographing according to an embodiment of the present invention will be described in detail with reference to FIGS. 2 to 3.

2 is a flowchart of a conventional management service (expiration date application) for a purchased product.

Referring to FIG. 2, a user must purchase a product, check a receipt for the purchased product, and manually repeatedly input the user directly to the user terminal 200, and the information input from the user is a server ( 100) can be stored.

In other words, it is a service that simply provides information manually input by the user to the server 100 through the terminal and, when requesting the information, only displays the information input by the user.

On the other hand, referring to FIG. 3, a service flow of a method for automatically storing food information in a refrigerator based on OCR through receipt photographing according to an embodiment of the present invention can be checked.

When a specific user purchases a product, checks the receipt for the purchased product, and photographs the receipt through the camera module of the user terminal 200, the user terminal 200 can receive an image of the receipt from the camera module. have.

For the received image, by applying OCR technology, a first list of which products and how many were purchased can be created, and the expiration date corresponding to each product can be searched and extracted from the database of the server 100. have.

After extracting the expiration date corresponding to each product, a second list including the expiration date in each item of the first list can be created and provided to the user, and the server 100 can be used to store data. Can provide.

The server 100 may collect a user-customized recipe and provide it to the user terminal 200 based on the second list provided from the user terminal 200 and food preference information of the user previously registered, By providing the second list to the cooking expert, a recommended meal may be provided to the user terminal 200 through the expert, or a service for managing the recommended meal may be provided.

In addition, a user who has a user terminal 200 in which an OCR-based refrigerator food information automatic storage program is stored through receipt photographing, and a specialist who has been provided with a second list of the corresponding user, automatically receives OCR-based refrigerator food information through receipt photographing. Through the storage platform, mutual communication may be possible.

Hereinafter, with reference to FIG. 4, the OCR processing and data storage process performed in the user terminal 200 of the OCR-based refrigerator food information automatic storage method through receipt photographing according to an embodiment of the present invention will be described in detail. .

The user terminal 200 may collect data by receiving an image in which a purchase receipt is photographed through a camera module.

As shown in FIG. 4, by correcting the receipt image in which the product name and number are written, a first list consisting of the product name and the number of products may be generated.

After the first list is created, a second list is generated through OCR processing, but the expiration date information corresponding to the product name can be searched and extracted from the database of the server 100 and included, and the expiration date information is included in the server ( It is information previously stored in 100), and may be stored as an average value for each product.

The second list may be configured including an item, quantity, and expiration date for each item, and the second list may be provided to a user, and data may be classified and stored in the server 100.

Hereinafter, a method for automatically storing food information in an OCR-based refrigerator through photographing a receipt according to an embodiment of the present invention will be described with reference to FIG. 5.

According to an embodiment of the present invention, the server 100 may collect preference information for the user's food through the user terminal 200 in advance, and the expiration date for a number of unspecified products (food ingredients and foods, etc.) And information on a recipe for food using a plurality of unspecified products may be collected and stored as big data.

The user terminal 200 may include a camera module, and may receive an image of a receipt through the camera module (S610).

By correcting the received image, the product name and the number of products may be collected, and a first list including the product name and the number of products may be generated from the corrected image (S620).

After the first list is generated, expiration date information corresponding to each product name may be searched and extracted from the database of the server 100 based on the product name of the first list (S630).

Thereafter, the user terminal 200 may generate a second list including the expiration date information extracted in step (S630) in the first list, and a second list including the product name, number, and expiration date to the user. Can provide.

In addition, based on the second list, the server 100 may search for a recipe containing a product corresponding to the product name and provide it to the user terminal 200, and at this time, the user's food preference information may be considered. (S640).

In providing or recommending a recipe, it may be performed in consideration of the user's preference for food and the expiration date, and a recipe containing a product with an imminent expiration date may be preferentially provided or recommended.

In addition, the second list may be provided to an expert terminal (not shown), and a recipe may be provided from an expert terminal through the user terminal 200, or a recommended diet may be provided, and inquiries may be made through real-time conversation or postings and comments. Communication such as response may be possible.

An embodiment of the present invention may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Further, the computer-readable medium may include all computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

Although the methods and systems of the present invention have been described in connection with specific embodiments, some or all of their components or operations may be implemented using a computer system having a general-purpose hardware architecture.

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains will be able to understand that other specific forms can be easily modified without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative and non-limiting in all respects. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. do.

10: system
100: server 200: user terminal

Claims (2)

In the OCR-based refrigerator food information automatic storage method through receipt photographing performed by a terminal,
(a) receiving an image photographing a purchase receipt for one or more products from a camera module of a user terminal;
(b) generating a first list including one or more product names and the number of products in the image;
(c) retrieving the expiration date of the product corresponding to each product name in the first list from the database, and generating a second list including one or more product names, the number of products, and the product expiration date; And
(d) retrieving and extracting a food recipe including each product from the database based on the second list and a diet in consideration of user preference; Including,
The database is data on expiration date, statistics on purchased products, user's food preference and customized diet management,
OCR-based refrigerator food information automatic storage method through receipt photographing.
The method of claim 1,
Before step (a),
To further comprising the step of receiving input of the user's food preference information, allergy information, and health information,
OCR-based refrigerator food information automatic storage method through receipt photographing.

Images