JP2006126945A - Component collation program and server device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable a consumer to easily confirm components contained in a commodity by means of a commodity label. <P>SOLUTION: Allergens that the commodity contains are printed using a digital watermark on the commodity label. By taking a picture of the commodity label using a cell phone, a consumer can confirm whether or not the commodity contains the allergens that cause allergy symptoms in the consumer. To achieve this function of confirming allergens, the cell phone extracts from the image data of the commodity label the allergens recorded using the digital watermark and collates the allergens with the allergens previously registered by the consumer. By simply taking a picture of the commodity label using the cell phone, the consumer can confirm whether or not the commodity contains the allergens peculiar to himself or herself, whereby the consumer is relieved of the burden of making confirmation. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a component verification program and a server device, for example, relates to a printer that prints an allergen on a label and compares it with a pre-registered allergen.

In recent years, interest in food allergies has increased.
Food allergy develops when a person with an allergic condition consumes a food containing an allergen protein.
There are a wide variety of allergens such as buckwheat, soy, wheat, and eggs, and allergens that cause allergic symptoms vary from person to person.

In particular, after processed foods are manufactured from various raw materials and become products, it is difficult to determine from the appearance what raw materials are contained.
Therefore, in order to prevent people with allergies from accidentally eating processed foods that contain allergens, it is used on labels and packaging of processed foods, such as “This food contains soba”. It has been done to display allergens. Also, some allergens are required to be labeled by law.
As a technique for presenting allergens used in this way to consumers, there is the following label printer.

JP 2002-361194 A

  In this technique, allergens contained in food are clearly printed at predetermined positions on a product label. The consumer can confirm the allergen contained easily by checking the label.

However, in recent years, the number of allergens that are obligated to be presented has been increasing, while the printing area of product labels is limited. In addition, it is desirable to print allergens that do not require display in order to obtain consumer comfort.
If these allergens are printed on a product label in addition to product information such as product name and content, a large amount of characters must be printed in thin letters, making the label difficult for consumers to read.

  Accordingly, an object of the present invention is to enable a consumer to easily confirm the components contained in a product using a product label.

In order to achieve the above object, according to the present invention, in the invention described in claim 1, a product storage function for storing a component specified in advance by a user and a product label printed with component information for specifying a product-containing component Component information extraction function for extracting the component information from the image data obtained by photographing the image, detection function for detecting whether or not the component specified by the extracted component information is included in the stored component, and the detection function And a component verification program for realizing a notification function for notifying the detection result by the computer.
According to a second aspect of the present invention, the component information is printed by invisible information that can be extracted by predetermined image processing, and the component information extraction function performs the image processing on the acquired image data. The component verification program according to claim 1, wherein the component information is extracted.
According to the third aspect of the present invention, the component storage function for storing the component designated in advance by the user and the image data obtained by photographing the product label on which the component information for specifying the component contained in the product is printed are transmitted to a predetermined server device An image data transmission function to be performed; a component reception function to receive component information included in the transmitted image data from the server device; and a component specified by the extracted component information is included in the stored component. There is provided a component collation program that realizes a detection function for detecting whether or not an image is detected and a notification function for notifying a detection result by the detection function by a computer.
The invention according to claim 4 includes a list presentation function for presenting a list of components, and a selection reception function for accepting selection of a component by a user from the presented list, wherein the component storage function is selected. The component verification program according to claim 1, 2, or 3 is provided.
In the invention according to claim 5, image data receiving means for receiving image data obtained by photographing a product label on which component information for specifying a component included in the product is printed from the terminal device, and the component from the received image data. There is provided a server device comprising: component information extracting means for extracting information; and component information transmitting means for transmitting the extracted component information to the terminal device.
In the invention described in claim 6, the component storage function for storing the component designated in advance by the user and the image data obtained by photographing the product label on which the component information for specifying the component contained in the product is printed are received from the terminal device. Image data reception means, component information extraction means for extracting the component information from the received image data, and detection for detecting whether or not the component specified by the extracted component information is included in the stored component There is provided a server apparatus comprising: a means; and a result transmission means for transmitting a detection result by the detection means to the terminal device.

  According to the present invention, the consumer can easily confirm the components contained in the product using the product label.

(1) Outline of Embodiments Information for identifying allergens contained in a product (hereinafter simply abbreviated as “allergen”) is printed on the product label with a digital watermark. If this product label is photographed, it can be confirmed whether or not the product contains an allergen that causes the consumer to develop allergic symptoms.

  In order to realize this allergen confirmation function, this mobile phone has a function of extracting allergens recorded with electronic watermarks from photographed image data of product labels and checking them with allergens registered in advance by consumers. Yes.

In this embodiment, the allergen is printed on the product label with a digital watermark, so the allergen can be printed over the normal display content such as the product name and content, and the limited print surface of the product label can be printed. It can be used effectively.
In addition, the consumer can confirm the presence or absence of an allergen peculiar to the consumer only by photographing the product label with the mobile phone, thereby reducing the burden of confirmation on the consumer.

(2) Details of Embodiment FIG. 1 is a block diagram illustrating an example of a system configuration of an allergen management system 1 according to the present embodiment.
The allergen management system 1 includes a server 2, a printer 3, a product label 4, a mobile phone 5, and the like.

The printer 3 is installed in a store that sells products, and issues a product label.
The printer 3 has a network connection function for connecting to a network such as the Internet, and downloads and stores print information from the server 2 via the network.

  The print information includes a product code for identifying the product, display information for displaying on the product label, and allergen information for printing with a digital watermark. The printer 3 prints the product label using the print information. To do.

FIG. 2 is a diagram illustrating an example of a logical configuration of print information.
As shown in FIG. 2, the print information includes a product code, display information, allergen information, and other information.
The product code is identification information uniquely assigned to a product sold at a store, and the product can be specified by the product code.

The display information includes, for example, information for explicitly printing on a product label such as a product name, content, logo mark, manufacturer, and the like.
The allergen information is composed of information for specifying the allergen contained in the product. For example, when the allergens included in the product are buckwheat and wheat, the allergen information is composed of information for specifying buckwheat and wheat.

  The printer 3 (FIG. 1) requests and acquires print information from the server 2 as appropriate, and always stores the latest print information. Then, when the person in charge at the store inputs the product code and the number of copies to the printer 3, the printer 3 prints the product label 4 using the print information specified by the product code.

  The product label 4 is printed with display information that can be visually recognized by the consumer, such as a product name, and allergen information that is read by the mobile phone 5. The printed product label 4 is printed by a person in charge of the store or the like. It is affixed to the product and placed in a state where the consumer can refer to it.

FIG. 3A shows an example of a product label printed by the printer 3. The example shown in FIG. 3 (a) is a product label issued for soba shochu, and the product includes the allergen buckwheat.
On the product label, allergen check compatible label display 10, trademark 11, content information 12 and the like are printed in a visible state. These are printed using the display information included in the print information.

The allergen check compatible label display 10 is for notifying the consumer that the product label can be confirmed by the mobile phone 5.
The consumer can know that the allergen information included in the product is printed on the product label 4 by digital watermark by looking at the label display 10 for allergen check.

The trademark 11 has “Kuramoto” and the name of the product printed in a unique font, and the consumer can recognize the product he / she intends to purchase using the trademark 11.
The content information 12 is printed with information such as name, distillation method, alcohol content, raw materials, characteristics, content, expiration date, brewery, etc., and the consumer can know the details of the product from these information.

Further, allergen information is embedded in the product label by digital watermark, and this is shown in FIG.
As shown in FIG. 3B, allergen information 13 of “soba” is printed on the product label by digital watermark.

  In FIG. 3B, the allergen information 13 is shown so that it can be visually recognized. However, in practice, it is embedded in the background image so as not to be visually recognized or difficult to visually recognize with a digital watermark. It is.

In the example of FIG. 3B, the name of the allergen (here, “soba”) is printed as it is as the allergen information 13, but ID information associated with the allergen may be printed.
For example, an ID number may be associated with each allergen, and this ID number may be embedded in a product label with a digital watermark.
Further, when the font is not standard, the character can be recognized from the similarity of the characters.

Various methods for digital watermarking have been proposed for binary images and grayscale images. In the present embodiment, since digital watermarks are printed on product labels, those for binary images are suitable. Yes.
As a digital watermarking method for a binary image, a method using a density pattern method, a systematic dither method, a difference diffusion method, or the like is known.

  In this embodiment, in order to effectively use the printing surface of the product label, the visible information (display information) is overlaid with the invisible information (allergen information). However, the allergen information is converted into visible information. For example, allergen information can be printed on a product label using a barcode or the like.

  As described above, the printer 3 (FIG. 1) receives the print information including the display information and the allergen information from the server 2 and stores it, and further displays the allergen information on the product label with the digital watermark. Can be printed on top of each other.

Next, the mobile phone 5 (FIG. 1) will be described.
The mobile phone 5 has a camera function and an Internet connection function.
The mobile phone 5 constitutes a computer having a storage device such as a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory).

Further, the mobile phone 5 is installed with an application program that causes the CPU (Central Processing Unit) to extract the allergen information from the product label and to check the allergen registered by the consumer in advance.
This application program is a program in which a consumer connects the mobile phone 5 to the server 2, downloads it from the server 2, and installs it on the mobile phone 5.

FIG. 4 is a diagram showing an example of the appearance of the mobile phone 5.
The mobile phone 5 includes a camera 20, a display unit 21, operation keys 22, an input key 23, and the like.

  As will be described later, the mobile phone 5 includes an information processing unit including a CPU, a ROM, a RAM, and the like, and an EEPROM (Electrically Erasable and Programmable ROM), for example, and stores various programs and data. Built-in readable / writable storage media.

  The camera 20 includes an optical system including a lens and the like, and a CCD (Charge Coupled Device) that converts an image of a subject projected by the optical system into digital data. A consumer takes a product label with the camera 20. The contents (display information and allergen information) printed on the product label can be converted into photographed image data.

  The display unit 21 is configured by a liquid crystal display, for example, and can display information related to a call such as a telephone number, and can display various screens transmitted from a website via the Internet on a browser.

The operation key 22 is a key for the customer to perform various screen operations on the screen displayed on the display unit 21 such as selecting an icon displayed on the display unit 21 or selecting a link. .
The input key 23 is a key for inputting numbers, characters, symbols, and the like to the mobile phone 5.

The display unit 21 in FIG. 4 displays a top screen when the application program is activated.
On the top screen, the title of the application is displayed as “allergen check”, so that the consumer can know that the application program has started.

On this screen, a check button 25, a registration button 26, an end button 27, and the like are displayed.
When the check button 25 is selected, the function of reading the allergen information of the product label and checking it with the self-allergen registered by the consumer is activated.
When the registration button 26 is selected, a function for the consumer to register his / her allergen in the mobile phone 5 is activated.
When the end button 27 is selected, the application program can be ended.

FIG. 5A is a diagram showing an example of a registration screen displayed on the display unit 21 when the registration button 26 is selected on the top screen.
On the registration screen, a list of allergens that can be registered is displayed along with an instruction to the consumer "Please select an allergen to register."
In the list of allergens, a selection marker 28 for selecting an allergen to be registered is also displayed.

The selection marker 28 can be shifted on each allergen by operating the operation key 22 or the like.
The consumer moves the selection marker 28 over the allergen to be registered, presses a predetermined key among the input keys 23, selects the allergen, and stores it in the storage device of the mobile phone 5.
The allergen registered here becomes the object of collation with the allergen information of the product label.

When the consumer selects one or a plurality of allergens that cause allergic symptoms and then presses a predetermined key of the input keys 23 to end the registration process, a registration confirmation screen as shown in FIG. It is displayed on the display unit 21.
The example of FIG. 5 (b) shows that buckwheat has been registered as an allergen. In addition to buckwheat, multiple allergens such as soybeans and wheat can be registered.

FIG. 6A is a diagram showing an example of a check screen displayed on the display unit 21 when the check button 25 is selected on the top screen.
On the check screen, a message such as “Please shoot an allergen check label” is displayed to prompt the consumer to shoot the product label.

According to this display, when the consumer operates the mobile phone 5 to photograph the product label, a result screen as shown in FIG. 6B is displayed.
In the example of FIG. 6B, the detected allergen “Soba” is displayed following the display “Next allergen has been detected”.
Further, if the allergen information of the product label does not include the allergen registered by the consumer, “Registered allergen was not detected” or the like is displayed.

  Various other methods for notifying the detection result can be considered. For example, when an allergen is detected, it is possible to generate a sound from a speaker included in the mobile phone 5 or to notify the consumer by vibrating the mobile phone 5 with a vibrator.

FIG. 7 is a block diagram illustrating an example of a functional configuration of the mobile phone 5.
As shown in FIG. 7, the mobile phone 5 includes a user interface 31, an allergen list download unit 32, an allergen list storage unit 33, an allergen registration unit 34, a registered allergen storage unit 35, a collation unit 36, an image analysis unit 37, an image The imaging unit 38 and the like are included.

  Among these components, the allergen list download unit 32 to the image analysis unit 37 configuring the module 39 are configured by executing an application program on the CPU of the mobile phone 5.

  The user interface 31 includes, for example, an output device such as a display unit 21 (FIG. 4), a speaker, and a vibrator, and input devices such as operation keys 22, input keys 23, and a microphone. This is a functional unit that allows the consumer to perceive information input to the mobile phone 5 or information output from the mobile phone 5.

The allergen list download unit 32 is a functional unit that downloads from the server 2 a list of allergens from which consumers select.
The type of allergen that needs to be verified may change, and the consumer occasionally accesses the server 2 to download the allergen list from the server 2.
The allergen list downloaded here is used to display a list of allergens on the registration screen shown in FIG.

  The allergen list storage unit 33 is a functional unit that stores the allergen list downloaded from the server 2. More specifically, the allergen list downloaded by the allergen list download unit 32 is stored in a predetermined storage area of a storage device such as an EEPROM.

  The allergen registration unit 34 presents a list of allergens to the consumer on the registration screen using the allergen list stored by the allergen list storage unit 33 and accepts the allergen selection by the consumer.

  The registered allergen storage unit 35 stores the allergen registered by the consumer in the allergen registration unit 34 (hereinafter, registered allergen) in a predetermined area of a storage device such as an EEPROM. The registered allergen storage unit 35 provides the registered allergen stored in the storage device to the verification unit 36 when allergen verification is performed.

The image photographing unit 38 includes the camera 20 of the mobile phone 5 and converts a subject photographed by the camera 20 into photographed image data composed of digital data.
When a consumer photographs a product label using the image photographing unit 38, photographed image data including allergen information composed of display information and a digital watermark is generated.

The image analysis unit 37 acquires photographed image data obtained by photographing the product label from the image photographing unit 38, extracts the allergen information through image processing.
Then, the image analysis unit 37 outputs the extracted allergen information to the collation unit 36.

The matching unit 36 acquires the registered allergen from the registered allergen storage unit 35, acquires allergen information from the image analysis unit 37, and verifies them.
More specifically, the matching unit 36 determines whether there is a match between the list of allergens specified by the allergen information and the list of allergens specified by the registered allergen.

  If there is a match, the verification unit 36 presents the allergen to the consumer through the user interface 31, and if there is no match, the matching unit 36 informs the user interface 31 that there is no match. Present it to the consumer.

FIG. 8 is a block diagram showing an example of the hardware configuration of the mobile phone 5.
The cellular phone 5 is configured by connecting each functional unit such as a CPU 40, a RAM 41, a ROM 42, an input / output unit 43, a display unit 44, a communication control unit 45, and a storage unit 47 through a bus line 46.
In addition, this block diagram shows a part necessary for collation of allergens in the configuration of the mobile phone 5, and does not show the configuration related to the call function.

The CPU 40 is a central processing unit that performs various types of arithmetic processing, information processing, and control of each component constituting the mobile phone 5 according to a predetermined program.
In the present embodiment, for example, an application program is downloaded from the server 2 and installed in the mobile phone 5, or the application is executed to extract allergen information from the photographed image data of the product label and compare it with a registered allergen. To do.

The ROM 42 is a read-only memory that stores basic programs and data for operating the mobile phone 5.
The RAM 41 is a readable / writable memory that provides a working area for the CPU 40 to operate.

The input / output unit 43 is a functional unit for inputting / outputting information to / from the mobile phone 5 and includes an input unit, an output unit, and the like.
Examples of the input unit include the camera 20, the operation key 22, the input key 23, and a microphone that detects a consumer's voice shown in FIG.
The output unit includes the display unit 21 shown in FIG. 4, a speaker that emits sound to the consumer, and a vibrator that transmits vibration to the consumer.

  The communication control unit 45 is a functional unit that connects the mobile phone 5 to the Internet. The mobile phone 5 can connect to the server 2 and other server devices via the communication control unit 45 to perform communication. The destination website is specified by the URL.

The storage unit 47 is a readable / writable storage device configured by, for example, an EEPROM.
The storage unit 47 includes a program unit 48 that stores programs and a data unit 49 that stores data.

The program unit 48 stores an application program downloaded from the server 2 and installed, and is provided from an OS (Operating System) that is a basic program for causing the mobile phone 5 to function or a server device. The browser software for using the service is stored.
The data unit 49 stores an allergen list downloaded from the server 2, a list of registered allergens registered by the consumer, and the like.

FIG. 9A is a diagram conceptually showing the structure of the allergen list stored in the data section 49.
As shown in FIG. 9A, allergens to be detected are recorded in the allergen list, such as “soba, soybean, wheat, egg, milk,. The consumer selects his / her allergen from these.

  In this example, the list is configured by writing the name of each allergen in characters, but the allergen list can be configured by a list of identification information associated with each allergen, for example.

FIG. 9B is a diagram conceptually showing the configuration of the registered allergen stored in the data unit 49.
Registered allergens consist of those selected by the consumer from those listed on the allergen list.

  In the example of FIG. 9B, “soba, egg” is selected. For this reason, the mobile phone 5 confirms whether or not the allergen information included in the product label includes at least one of buckwheat and egg.

  In the present embodiment, the registered allergen is separately created from the allergen list. For example, the allergen selected by the consumer by flagging the allergen in the allergen list can be identified by the allergen list. You can also

FIG. 10 is a flowchart for explaining a procedure for the mobile phone 5 to confirm allergen information included in the product label. In the following description, the operation will be described corresponding to each functional unit illustrated in FIG.
First, after starting an application on the mobile phone 5, the consumer takes a product label with the camera 20 in accordance with an instruction of the application.

Then, the image photographing unit 38 converts the image of the product label into digital data and obtains photographed image data (step 5).
Next, the image analysis unit 37 performs image processing on the captured image data to extract digital watermarked allergen information (step 10). The extracted allergen information is transferred from the image analysis unit 37 to the collation unit 36.

The collation unit 36 acquires allergen information from the image analysis unit 37 and further acquires a registered allergen from the registered allergen storage unit 35.
And the collation part 36 collates whether the allergen described in the allergen information contains what matches a registration allergen (step 15).

When there is a matching allergen (step 20; Y), the collation unit 36 displays the matched allergen on the user interface 31 to the consumer (step 25), and ends the process.
When there is no matching allergen (step 20; N), the collation unit 36 notifies the consumer that there is no matching allergen in the user interface 31 (step 30), and ends the process.

(Modification 1)
Although the embodiment has been described above, various modifications are possible. Next, Modification 1 will be described with reference to FIG.
In the present modification, the configuration other than the server 2 and the mobile phone 5 is the same as that of the above-described embodiment, so that the description thereof will be omitted and only the server 2 and the mobile phone 5 will be described.

In the present modification, when the mobile phone 5 captures the product label 4, it transmits the captured image data to the server 2.
The server 2 performs image processing on the captured image data transmitted from the mobile phone 5 and extracts allergen information. Then, the extracted allergen information is transmitted to the mobile phone 5.

The cellular phone 5 receives the allergen information from the server 2 and collates it with a registered allergen stored in advance. Then, the mobile phone 5 notifies the consumer of the collation result.
Thus, in this modification, since the image processing is performed by the server 2, the load on the mobile phone 5 is reduced, and the mobile phone 5 with low information processing capability can be implemented.

(Modification 2)
Next, Modification 2 will be described with reference to FIG. In this modification, registered allergens for each consumer are managed by the server 2 and allergen verification is performed by the server 2.

FIG. 12 is a diagram showing an example of the logical configuration of the registered allergen database stored in the server 2.
As shown in the figure, each consumer is assigned a unique user ID, and a registered allergen for each consumer is stored.
For example, a consumer whose user ID is “001” has registered soba noodles. Allergen registration can be performed on a website operated by the server 2.

The cellular phone 5 transmits the photographed image data of the product label and the user ID to the server 2.
When the server 2 receives the captured image data and the user ID from the mobile phone 5, the server 2 first extracts allergen information from the captured image data.

Then, a match between the allergen identified by the extracted allergen information and the registered allergen registered for the user ID is detected, and if there is a matching allergen, the allergen is transmitted to the mobile phone 5, If there is no matching allergen, the mobile phone 5 is notified accordingly.
The consumer can check the detection result from the server 2 with the mobile phone 5 and know whether or not the registered allergen is included in the product.

While the present embodiment and its modifications have been described above, various methods of use can be considered for product labels.
For example, a product label can be affixed to a food sample in a cafeteria, and the consumer can check for the presence of allergens when ordering food.
Furthermore, the product label can be applied not only to food products but also to products that consumers may come into contact with allergens, such as cosmetics, soap, and shampoos.

The following effects can be obtained by the embodiment and the modification described above.
(1) A consumer can confirm whether or not an allergen is contained in a product simply by photographing the product label with a mobile phone.
(2) Since the allergen information is digitally watermarked and printed over the display information, the display area of the product label can be used effectively.
(3) The consumer can select an allergen to be applied to him from the allergen list.
(4) By notifying the confirmation result of the product label by voice or vibration, the product label can be easily confirmed even by a low vision person.

It is the block diagram which showed an example of the system configuration | structure of an allergen management system. FIG. 4 is a diagram illustrating an example of a logical configuration of print information. It is the figure which showed an example of the product label. It is the figure which showed an example of the external appearance of a mobile telephone. It is the figure which showed an example of the screen in the case of registering an allergen. It is the figure which showed an example of the screen in the case of checking an allergen. It is the block diagram which showed an example of the functional structure of a mobile telephone. It is the block diagram which showed an example of the hardware structure of a mobile telephone. It is the figure which showed an example of the data memorize | stored in the data part. It is a flowchart for demonstrating the procedure in which a mobile telephone confirms allergen information. It is a block diagram for demonstrating a modification. It is the figure which showed an example of the logical structure of the registration allergen database used by a modification.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Allergen management system 2 Server 3 Printer 4 Product label 5 Mobile phone 31 User interface 32 Allergen list download part 33 Allergen list storage part 34 Allergen registration part 35 Registered allergen storage part 36 Collation part 37 Image analysis part 38 Image photographing part 39 Module

Claims (6)

A component storage function for storing the components designated by the user in advance;
A component information extraction function for extracting the component information from image data obtained by photographing a product label on which component information for specifying a component included in the product is printed;
A detection function for detecting whether or not the component specified by the extracted component information is included in the stored component;
A notification function for notifying the detection result by the detection function;
Is a component verification program that realizes   The component information is printed by non-visible information that can be extracted by predetermined image processing, and the component information extraction function extracts the component information by performing the image processing on the acquired image data. The component verification program according to claim 1. A component storage function for storing the components designated by the user in advance;
An image data transmission function for transmitting image data obtained by photographing a product label on which component information for specifying a component included in the product is printed, to a predetermined server device;
A component receiving function for receiving component information included in the transmitted image data from the server device;
A detection function for detecting whether or not the component specified by the extracted component information is included in the stored component;
A notification function for notifying the detection result by the detection function;
Is a component verification program that realizes A list display function for displaying a list of ingredients;
A selection acceptance function for accepting selection of a component by the user from the presented list,
4. The component verification program according to claim 1, wherein the component storage function stores the selected component. Image data receiving means for receiving image data obtained by photographing a product label on which component information for specifying a component included in the product is printed from the terminal device;
Component information extraction means for extracting the component information from the received image data;
Component information transmitting means for transmitting the extracted component information to the terminal device;
A server apparatus comprising: A component storage function for storing the components designated by the user in advance;
Image data receiving means for receiving image data obtained by photographing a product label on which component information for specifying a component included in the product is printed from the terminal device;
Component information extraction means for extracting the component information from the received image data;
Detecting means for detecting whether or not the component specified by the extracted component information is included in the stored component;
A result transmitting means for transmitting a detection result by the detecting means to the terminal device;
A server apparatus comprising:

Images