JP2018101353A - Determination system, method for determination, and determination program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow an appropriate assist to a user who performs work.SOLUTION: A determination system according to the present application includes an acquisition part and a determination part. The acquisition part acquires image information on a user who performs work including a plurality of phases of which operational contents are related to one another and of which sequence is determined in advance. The determination part determines a working phase, which is the phase in the work of the user, on the basis of the image information acquired by the acquisition part and the information on the phases.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a determination system, a determination method, and a determination program.

  2. Description of the Related Art Conventionally, there has been provided a technique for providing information to a user according to the work or situation that the user is performing. For example, a system that provides guidance information for performing guidance with augmented reality to a user is known.

JP 2012-014606 A

  However, in the above-described conventional technology, it may be difficult to enable appropriate assistance for a user who performs work. For example, when providing information using a paper medium possessed by the user, for example, appropriately providing information assistance to a user who is performing a task that is difficult to perform while holding the paper medium in his / her hand May be difficult.

  The present application has been made in view of the above, and an object thereof is to provide a determination system, a determination method, and a determination program that enable appropriate assistance for a user who performs work.

  The determination system according to the present application includes: an acquisition unit configured to acquire image information related to a user who performs a work including a plurality of phases in which work contents are associated with each other and the order between the phases is determined in advance; A determination unit that determines a working phase that is a phase in the user's work based on the image information acquired by the acquiring unit and information on the plurality of phases.

  According to one aspect of the embodiment, there is an effect that it is possible to enable appropriate assistance for a user who performs work.

FIG. 1 is a diagram illustrating an example of a determination process according to the embodiment. FIG. 2 is a diagram illustrating a configuration example of the determination system according to the embodiment. FIG. 3 is a diagram illustrating a configuration example of the determination apparatus according to the embodiment. FIG. 4 is a diagram illustrating an example of the work information storage unit according to the embodiment. FIG. 5 is a diagram illustrating an example of the output information storage unit according to the embodiment. FIG. 6 is a diagram illustrating a configuration example of the terminal device according to the embodiment. FIG. 7 is a flowchart illustrating an example of the determination process according to the embodiment. FIG. 8 is a sequence diagram illustrating an example of the determination process according to the embodiment. FIG. 9 is a diagram illustrating an example of work error notification in the determination system according to the embodiment. FIG. 10 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the determination device and the terminal device.

  Hereinafter, modes for carrying out a determination system, a determination method, and a determination program according to the present application (hereinafter referred to as “embodiments”) will be described in detail with reference to the drawings. Note that the determination system, the determination method, and the determination program according to the present application are not limited by this embodiment. In the following embodiments, the same portions are denoted by the same reference numerals, and redundant description is omitted.

(Embodiment)
[1-1. Determination process〕
An example of the determination process according to the embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of a determination process according to the embodiment. FIG. 1 shows a case where the work performed by the user is cooking, and the determination system 1 outputs information for assisting the cooking work by the user. For example, the example of FIG. 1 shows a case where the determination system 1 provides a cooking support service. Specifically, the example of FIG. 1 illustrates a case where the determination system 1 provides a dish support service related to a dish specified by the user. The work targeted by the determination system 1 is not limited to cooking, and may be any work as long as the work is performed by the user.

[1-2. (Configuration of judgment system)
First, the determination system 1 shown in FIGS. 1 and 2 will be described. As shown in FIG. 2, the determination system 1 includes a terminal device 10, a sensor device 20, and a determination device 100. The terminal device 10, the sensor device 20, and the determination device 100 are connected via a predetermined network N so as to be communicable by wire or wirelessly. FIG. 2 is a diagram illustrating a configuration example of the determination system according to the embodiment. The determination system 1 illustrated in FIG. 2 may include a plurality of terminal devices 10, a plurality of sensor devices 20, and a plurality of determination devices 100.

  The terminal device 10 is an information processing device used by a user. The terminal device 10 receives various operations by the user. Hereinafter, the terminal device 10 may be referred to as a user. That is, hereinafter, the user can be read as the terminal device 10. The terminal device 10 described above is realized by, for example, a smartphone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a PDA (Personal Digital Assistant), or the like.

  In the example shown in FIG. 1, the terminal device 10 has a function of a voice response system (voice recognition), detects voice information (hereinafter also referred to as “utterance”) uttered by the user, and uses the detected utterance as input information. Accept. For example, the terminal device 10 is a terminal device that processes a user's utterance. In the example of FIG. 1, when the terminal device 10 detects an utterance by the user, the terminal device 10 transmits input information based on the user's utterance to the distribution device 100 or displays content corresponding to the user's utterance.

  The sensor device 20 is an information processing device that detects predetermined information. The sensor device 20 may be a wearable terminal such as a glasses-type terminal or a wristwatch-type terminal. In the example of FIG. 1, the sensor device 20 is a glasses-type terminal worn by the user U1. In the example of FIG. 1, the sensor device 20 has an imaging function and a communication function. For example, the sensor device 20 has a camera function and a video function realized by an imaging element such as a CCD (Charge Coupled Device), and captures a still image or a moving image. In addition, the sensor device 20 transmits image information to the terminal device 10. The communication between the sensor device 20 and the terminal device 10 may be performed by, for example, a function of WiFi (registered trademark) (Wireless Fidelity) or Bluetooth (registered trademark), or other communication (short-range wireless communication or the like). This function may be performed.

  The determination apparatus 100 is an information processing apparatus that determines a working phase, which is a phase in a user's work, based on image information and information on a plurality of phases based on user behavior information. For example, the determination apparatus 100 may be an information processing apparatus that provides a dish assistance service.

  The determination apparatus 100 may have a voice recognition function. Further, the determination apparatus 100 may be able to acquire information from a voice recognition server that provides a voice recognition service. In this case, the determination system 1 may include a voice recognition server. In the example of FIG. 1, it is assumed that the determination device 100 and the voice recognition server use various conventional techniques as appropriate to recognize the user's utterance and specify the user who has spoken. Is omitted.

  From here, it returns to FIG. 1 and the determination process by the determination system 1 is demonstrated. The example of FIG. 1 shows a case where a user identified by the user ID “U1” (hereinafter, may be referred to as “user U1”) supports (assists) the work of making a dish “meat potato”. FIG. 1 shows a case where the user U1 uses a cooking support service by the determination system 1 at home or the like. In the following description, a phase determined by the determination system 1 as a phase in which the user is working may be referred to as a “working phase”.

  In FIG. 1, the determination device 100 will be described as determination devices 100-1 to 100-3 according to the timing at which determination is performed, the timing at which output is performed, and the like. That is, FIG. 1 shows that time has passed in the order of the determination devices 100-1, 100-2, and 100-3. The determination devices 100-1 to 100-3 are the same determination device 100. Hereinafter, the determination devices 100-1 to 100-3 will be referred to as the determination device 100 when they are not particularly distinguished.

  In the following, the work information storage unit 121 will be described as work information storage units 121-1 to 121-3 according to the working phase. That is, FIG. 1 shows that time has elapsed in the order of the work information storage units 121-1, 121-2, and 121-3. The work information storage units 121-1 to 121-3 are the same work information storage unit 121. Hereinafter, the work information storage units 121-1 to 121-3 will be referred to as the work information storage unit 121 when they are not particularly distinguished.

  Moreover, in FIG. 1, the terminal device 10 is demonstrated as terminal device 10-1 to 10-3. That is, FIG. 1 shows that time has elapsed in the order of the terminal devices 10-1, 10-2, and 10-3. The terminal devices 10-1 to 10-3 are the same terminal device 10. Hereinafter, the terminal devices 10-1 to 10-3 will be referred to as the terminal device 10 when they are described without particular distinction.

  For example, in the example of FIG. 1, the hatched phase in the work information storage unit 121 is a phase determined to be the working phase. For example, in the work information storage unit 121-2, since the second phase is hatched, it indicates that the working phase in the work of making the meat potato of the user U1 is the second phase. In FIG. 1, the working phase is illustrated by hatching for the sake of explanation, but the work information storage unit 121 may store information such as a flag for identifying the working phase.

  Further, the work information storage unit 121 in FIG. 1 indicates that the dish identified by the dish ID “CK1” is the dish name “meat potato”. Further, the work information storage unit 121 in FIG. 1 indicates that the dish identified by the dish ID “CK1” includes a plurality of phases including the first phase to the third phase. Further, the work information storage unit 121 in FIG. 1 indicates that the content of the first phase of the dish identified by the dish ID “CK1” is “align the ingredients”. Further, the work information storage unit 121 in FIG. 1 indicates that the target of the first phase of the dish identified by the dish ID “CK1” is “meat, potatoes ...”. As described above, the work information storage unit 121 supports the user's cooking such as information on a plurality of phases in which work contents are associated with each other and the order between the phases is determined in advance. Various kinds of information are stored. For example, as shown in the work information storage unit 121 in FIG. 1, each phase is associated (tagged) with the work content to be performed in that phase. In addition, for example, the order of the phases is determined in advance, such as the first phase to the twentieth phase. In this case, the user completes the desired dishes by performing the first phase, the second phase, the third phase, the fourth phase,..., The 20th phase and the numbers indicating the phases in ascending order. Work information is composed. Thus, the user performs the second phase after performing the first phase, performs the third phase after performing the second phase, performs the 20th phase after performing the 19th phase, and so on. By doing the work in order, you can cook. Hereinafter, a process in which the determination system 1 outputs information for assisting the cooking work by the user based on the work information storage unit 121 will be described.

  First, in the example of FIG. 1, when the user U1 cooks in the kitchen KC, the utterance PA11 “make meat potatoes” is performed around the terminal device 10. Thereby, the terminal device 10-1 acquires the utterance PA11 “Meat potato is made” as an input (step S11). For example, in the example of FIG. 1, the terminal device 10-1 is arranged around the kitchen KC that can detect the utterance of the user U <b> 1, and is an application related to a cooking support service (hereinafter also referred to as “cooking support app”) by the determination system 1. ) Is activated.

  Note that the user U1 may activate the cooking support application by pressing an icon of the cooking support application displayed on the terminal device 10 or performing a predetermined utterance. For example, the user U1 may activate the application by performing a predetermined utterance that activates an application such as “activate cooking support application XX”, “yes, XX”, or “ok, XX”. And the terminal device 10-1 transmits the audio | voice information of acquired utterance PA11 to the determination apparatus 100 as input information (step S12).

  Note that the terminal device 10 may transmit the speech information of the utterance PA11 to the speech recognition server, acquire the character information of the utterance PA11 from the speech recognition server, and transmit the acquired character information to the determination device 100. In this case, the terminal device 10 may transmit information indicating that the user who performed the acquired utterance PA11 is the user U1 to the determination device 100. When the terminal device 10 has a voice recognition function, the terminal device 10 may transmit only input information that needs to be transmitted to the determination device 100 to the determination device 100.

  The determination apparatus 100 that has acquired the voice information of the utterance PA11 from the terminal device 10 determines whether the utterance PA11 includes information related to the designation of a dish (hereinafter also referred to as “dish designation information”) (step S13). For example, the determination apparatus 100 determines whether the utterance PA11 includes cooking designation information by analyzing the voice information of the received utterance PA11. For example, the determination apparatus 100 determines whether or not dish designation information is included in the utterance PA11 by comparing a list of dish names targeted in the dish assistance service with a character string included in the utterance PA11. Note that the determination device 100 may determine whether or not the dish designation information is included in the user's utterance, or may be performed by another device. When the other device determines whether the user's utterance includes the dish designation information, the determination apparatus 100 provides the determination result whether the utterance PA11 includes the dish designation information or information on the designated dish name. Shall be obtained from the device.

  In the example of FIG. 1, the determination device 100 converts the received speech information of the utterance PA11 into character information, and indicates that the user U1 is trying to make meat potatoes in the utterance PA11 based on the converted character information. Is determined to be included. For example, the determination apparatus 100 identifies that the user U1 is trying to make meat potatoes by appropriately analyzing the character information of the utterance PA11 using various conventional techniques such as syntax analysis. That is, the determination apparatus 100 receives the dish designation information related to the dish name “meat potato” from the user U1 who has made the utterance PA11.

  For example, as illustrated in the work information storage unit 121-1 in FIG. 1, the determination device 100 uses the utterance PA 11 of the user U 1 as a target for the dish name “meat potato” identified by the dish ID “CK 1”. Provides information on For example, the determination device 100 provides the output information stored in the output information storage unit 122 to the terminal device 10.

  In the first phase, when the user U1 makes meat potatoes, the output information storage unit 122 in FIG. 1 outputs output information (such as “Please prepare meat, potatoes ...” identified by the output ID “OP1”). Hereinafter, this is also referred to as “first phase information OP1”. In addition, when the user U1 makes meat potatoes, the output information storage unit 122 in FIG. 1 outputs information such as “Please cut meat into bites” identified by the output ID “OP2” in the second phase. (Hereinafter also referred to as “second phase information OP2”).

  Hereinafter, the output information storage unit 122 will be described as output information storage units 122-1 to 122-3 in accordance with the output information to be provided. The output information storage units 122-1 to 122-3 are the same output information storage unit 122. In the following description, the output information storage units 122-1 to 122-3 will be referred to as the output information storage unit 122 when they are not particularly distinguished.

  In the example of FIG. 1, the determination apparatus 100 determines that the dish made by the user U1 is meat potato as shown in the work information storage unit 121-1, and therefore sets the first phase of meat potato as the working phase. Then, the determination device 100 provides output information corresponding to the working phase to the terminal device 10-1 (step S14). In the example of FIG. 1, as shown in the output information storage unit 122-1, the determination apparatus 100 has the first content “Please prepare meat, potatoes” corresponding to the first phase that is the working phase. The phase information OP1 is provided to the terminal device 10.

  And the terminal device 10 which received 1st phase information OP1 outputs 1st phase information OP1 as an audio | voice (step S15). Note that the terminal device 10 may output sound through a speaker. When the user U1 wears the earphone, the earphone transmits the first phase information OP1 to the earphone so that the earphone can receive the first phase information. OP1 may be output as sound. That is, the first phase information OP1 may be output as audio in any form as long as it can be transmitted as audio to the user U1. Further, the terminal device 10 may display the first phase information OP1 on the screen as character information.

  Then, the user U1 who has recognized the work contents of the first phase with the voice “please prepare meat, potatoes ...” included in the first phase information OP1 prepares meat and potatoes according to the contents of the sounds. It becomes. In the example of FIG. 1, the user U1 wears a sensor device 20 that is a glasses-type terminal, and the sensor device 20 captures an image using an imaging function. That is, the sensor device 20 images the user's line-of-sight direction. In addition, since the sensor device 20 is a glasses-type terminal, when the user performs a work using his / her hand, the user's hand being watched by the user is imaged.

  The sensor device 20 may capture a moving image, or may capture an image at a predetermined interval. In addition, the sensor device 20 transmits the captured image information to the terminal device 10. The sensor device 20 may transmit all the captured information to the terminal device 10, or may transmit image information extracted at a predetermined frame rate to the terminal device 10 when capturing a moving image.

  FIG. 1 shows a state where the user U1 (user U1 in the center of FIG. 1) has prepared meat MT1, potato PT1, and onion ON1, and the sensor device 20 uses image information IM11 obtained by imaging the situation. It transmits to the terminal device 10-2. Then, the terminal device 10-2 receives the image information IM11 (Step S16). For example, the terminal device 10 may receive the image information IM11 transmitted from the sensor device 20 by instructing the sensor device 20 to transmit the image information. Thus, the sensor device 20 may transmit the image information IM11 to the terminal device 10 in response to an instruction from the terminal device 10. Thereafter, the terminal device 10-2 transmits the image information IM11 to the determination device 100 (step S17). Note that the terminal device 10 may transmit image information extracted at a predetermined interval (frame rate or the like) to the determination device 100 when receiving all information captured by the sensor device 20 from the sensor device 20.

  The determination apparatus 100 that has acquired the image information IM11 from the terminal apparatus 10-2 determines the working phase of the user U1 (step S18). For example, the determination apparatus 100 determines the working phase of the user U1 based on the determination information “DCK1-1” (see FIG. 4) regarding the first phase of meat potato that is the working phase at that time.

  For example, the determination information “DCK1-1” includes the image information of the meat MT1, the image information of the potato PT1, and the image information of the onion ON1, and the determination device 100 acquires the image information acquired from the terminal device 10. By comparing IM11 and image information included in the determination information “DCK1-1”, it is determined whether or not the materials are available. In the example of FIG. 1, since the image information IM11 includes meat MT1, potato PT1, and onion ON1, the determination apparatus 100 determines that the user U1 has completed the first phase. That is, the determination apparatus 100 determines that the second phase, which is the next phase after the first phase, is the working phase. Note that the determination apparatus 100 may appropriately use various techniques of image processing such as pattern recognition as a technique for determining an object used for cooking from image information.

  In the example of FIG. 1, since the determination apparatus 100 determines that the working phase of the user U1 is the second phase as shown in the work information storage unit 121-2, the second phase of meat potato is set as the working phase. To do. Then, the determination device 100 provides output information corresponding to the working phase to the terminal device 10-2 (step S19). In the example of FIG. 1, as shown in the output information storage unit 122-2, the determination apparatus 100 corresponds to the second phase, which is the working phase, and the second phase with the content “Please cut the meat into bites”. Information OP2 is provided to the terminal device 10.

  And the terminal device 10 which received 2nd phase information OP2 outputs 2nd phase information OP2 as an audio | voice (step S20). For example, the terminal device 10 outputs the second phase information OP2 as sound by a speaker or an earphone worn by the user U1.

  Then, the user U1 who has recognized the work content of the second phase by the voice “Please cut the meat into a bite” included in the second phase information OP2 cuts the meat into a bite size according to the content of the voice. Become.

  The user U1 after step S20 in FIG. 1 (the user U1 on the right side in FIG. 1) shows a state in which the meat MT1 is cut into the meat MT1-1, MT1-2, MT1-3 on the cutting board CB, and the sensor device 20 The image information IM21 obtained by imaging the situation is transmitted to the terminal device 10. As described above, since the sensor device 20 is a glasses-type terminal, when the user U1 cuts the meat MT1 with a knife, the user's hand watched by the user U1 is captured. Therefore, the determination system 1 can collect image information used for determination of the working phase without causing the user U1 to be particularly conscious or extra effort.

  Then, the terminal device 10-3 receives the image information IM21 (step S21). Thereafter, the terminal device 10-3 transmits the image information IM21 to the determination device 100 (step S22).

  The determination apparatus 100 that has acquired the image information IM21 from the terminal device 10-3 determines the working phase of the user U1 (step S23). For example, the determination apparatus 100 determines the working phase of the user U1 based on the determination information “DCK1-2” (see FIG. 4) regarding the second phase of meat potato that is the working phase at that time.

  For example, the determination information “DCK1-2” includes image information in a state where the meat MT1 is formed into a bite, and the determination apparatus 100 determines whether the image information IM21 acquired from the terminal apparatus 10-3 and the determination information It is determined whether or not the meat MT1 is cut into a bite by comparing image information included in the information “DCK1-2”. In the example of FIG. 1, since the image information IM21 includes meat MT1-1, MT1-2, and MT1-3 that are cut into pieces, the determination device 100 has the user U1 as the second. Determine that the phase is complete. That is, the determination apparatus 100 determines that the third phase, which is the next phase after the second phase, is the working phase.

  In the example of FIG. 1, since the determination apparatus 100 determines that the working phase of the user U1 is the third phase as shown in the work information storage unit 121-3, the third phase of meat potato is set as the working phase. To do. Then, the determination device 100 provides output information corresponding to the working phase to the terminal device 10-3 (step S24). In the example of FIG. 1, as shown in the output information storage unit 122-3, the determination device 100 identifies “potatoes” identified by the output ID “OP3” that is output information corresponding to the third phase that is the working phase. The terminal device 10-3 is provided with output information (third phase information OP3) with the content “Please cut to bite size”. And the terminal device 10 which received 3rd phase information OP3 outputs 3rd phase information OP3 as an audio | voice. Thereafter, the determination system 1 repeats the process until all meat potato phases such as the third phase and the fourth phase are completed, that is, the user U1 finishes making meat potatoes.

  As described above, the determination system 1 collects information by the sensor device 20 worn by the user U1, thereby making the user U1 not particularly aware of the work in progress phase without extra effort. Image information used for the determination can be collected. As described above, the determination system 1 can appropriately grasp which phase the user U1 is performing by collecting image information from the sensor device 20 worn by the user U1. it can. Moreover, since the determination system 1 outputs the information regarding the work content corresponding to each phase in order, the user can work in each phase in order. Even if it is a dish that you do not know, the user can cook. In other words, each time the user completes a phase, the determination system 1 advances the phase sequentially to the next phase, and provides information on the work content corresponding to the phase, so that a user who does not normally cook, etc. Even so, cooking can be properly completed. Moreover, the determination system 1 outputs the output information corresponding to the work performed by the user U1 as a voice, so that the user U1 does not take his line of sight away from the work, while the user U1 is performing his / her work, Can recognize information. Accordingly, the determination system 1 can provide appropriate assistance to the user who performs the work by providing the user U1 with the output information corresponding to the in-work phase of the user U1.

[1-3. About work)
In the above-described example, cooking is shown as an example of the work. However, the work targeted by the determination system 1 is not limited to cooking, and any work may be performed as long as the work is performed by the user. For example, the determination system 1 may target various works in which the user performs physical actions. For example, the determination system 1 may target an operation performed by a user such as calligraphy or knitting using a hand. Further, the determination system 1 may be introduced into a predetermined factory or the like and applied to process management or the like at the factory.

[1-4. System configuration〕
In the example described above, the determination system 1 that transmits information from the terminal device 10 to the determination device 100 and determines the working phase in the determination device 100 is described as an example. However, the configuration of the determination system is not limited to the above. . For example, a configuration of a determination system in which the terminal device 10 determines a working phase may be used. That is, in this case, the terminal device 10 may be integrated with the determination device 100. Further, the terminal device 10 may be integrated with the sensor device 20. For example, the terminal device 10 may be integrated with the determination device 100 and the sensor device 20. For example, the user may be provided with a cooking support service by the determination system by the glasses-type terminal device 10 worn. In this case, the user may be provided with a cooking support service by the determination system using the stand-alone terminal device 10. In addition, for example, the terminal device 10 transmits a dish designation to an external device such as the determination device 100, receives work information and output information related to the designated dish from the external device such as the determination device 100, and receives the received work information or A cooking support service may be provided by the determination system based on the output information. Note that the above is an example, and the determination system may be configured in any way as long as the above-described service can be provided to the user.

[1-5. About the phase)
Further, in the above example, the case where the user U1 goes in order from the first phase to the third phase is shown, but the user U1 may change the order of the phases that may be out of order and perform the work. Also good. For example, in the example of FIG. 1, either the second phase for cutting meat or the third phase for cutting potatoes may be performed first. Therefore, the determination system 1 may identify the entire cutting operation as one phase, or identify each phase related to the cutting operation as an unordered phase. When the processing of each phase is out of order, the determination apparatus 100 may proceed to the next operation by setting the next phase as a working phase after completing the processing of each phase out of order. For example, the determination apparatus 100 may determine that the next phase, such as frying in a pan, is the in-operation phase when all the operations for cutting the second phase and the third phase are completed.

[1-6. About output information)
Moreover, although the case where the user U1 outputs one output information in the first phase to the third phase has been described in the above example, the determination system 1 may output a plurality of output information in each phase. For example, when a plurality of pieces of output information are output in each phase, the determination system 1 stores a plurality of pieces of output information in association with each phase, or sets an output order of a plurality of pieces of output information associated with each phase. Or a plurality of pieces of output information may be output in each phase.

[1-6-1. Warning and work reminder)
Moreover, the determination system 1 may output various information according to the user's work content. For example, the determination system 1 may output warning information when a predetermined period has elapsed without the transition to the next phase or the completion of work after the work phase is determined. For example, the determination system 1 may output information that prompts the user to proceed with work. For example, the determination system 1 may output information that prompts the user to proceed with work such as “Please proceed with work” when the user does not perform work for more than 5 minutes after the change in work phase. Further, the determination system 1 may output warning information when the user performs an operation different from the operation content in the operation phase. Details of this point will be described later.

[2. (Configuration of judgment device)
Next, the configuration of the determination apparatus 100 according to the embodiment will be described with reference to FIG. FIG. 3 is a diagram illustrating a configuration example of the determination apparatus according to the embodiment. As illustrated in FIG. 3, the determination apparatus 100 includes a communication unit 110, a storage unit 120, and a control unit 130. The determination apparatus 100 includes an input unit (for example, a keyboard and a mouse) that receives various operations from an administrator of the determination apparatus 100 and a display unit (for example, a liquid crystal display) for displaying various types of information. May be.

(Communication unit 110)
The communication unit 110 is realized by, for example, a NIC (Network Interface Card). The communication unit 110 is connected to a network (for example, the network N in FIG. 2) by wire or wireless, and transmits / receives information to / from the terminal device 10.

(Storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. As illustrated in FIG. 3, the storage unit 120 according to the embodiment includes a work information storage unit 121 and an output information storage unit 122.

(Work information storage unit 121)
The work information storage unit 121 according to the embodiment stores information related to work. For example, the work information storage unit 121 stores various types of information such as information on each phase included in the work. FIG. 4 is a diagram illustrating an example of the work information storage unit according to the embodiment. FIG. 4 shows an example of information related to work stored in the work information storage unit 121. As illustrated in FIG. 4, the work information storage unit 121 includes items such as “cooking ID”, “cooking name”, “phase”, “content”, “target”, and “determination information” as work-related information. .

  “Cooking ID” indicates identification information for identifying cooking as a work. “Cooking name” indicates the cooking name of the corresponding work. “Phase” indicates a phase included in the dish. “Content” indicates the work content of the corresponding phase. “Target” indicates the target of work in the corresponding phase. Further, “determination information” indicates information used for determining whether or not the operation of the corresponding phase has been completed.

  For example, in the example illustrated in FIG. 4, the dish identified by the dish ID “CK1” indicates the dish name “meat potato”. Further, for example, in the example illustrated in FIG. 4, the dish identified by the dish ID “CK1” includes a plurality of phases including the first phase to the third phase. Further, for example, in the example illustrated in FIG. 4, the first phase of the dish identified by the dish ID “CK1” indicates that the content is “align the ingredients”. Further, for example, in the example illustrated in FIG. 4, the first phase of the dish identified by the dish ID “CK1” indicates that the target is “meat, potatoes ...”. For example, in the example illustrated in FIG. 4, the determination information of the first phase of the dish identified by the dish ID “CK1” is the determination information “DCK1-1”.

  In the example of FIG. 4, the determination information is illustrated abstractly as “DCK1-1” or the like, but the determination information may include various information used for determining whether the corresponding phase is completed. . For example, when an image is used for phase determination, a still image, a moving image, text data, a URL (Uniform Resource Locator), or a file path name indicating the storage location thereof may be stored as determination information. For example, when an image is used for the determination in the first phase of the dish identified by the dish ID “CK1”, the determination information “DCK1-1” includes the image information of meat MT1, the image information of potato PT1, and the onion. Image information of ON1 may be included. In this case, for example, the determination apparatus 100 may determine whether the materials are prepared by comparing the image information acquired from the terminal apparatus 10 with the image information included in the determination information “DCK1-1”. . In addition, for example, when the phase is a phase having a designation related to time such as “simmer for 5 minutes”, the determination information may include information related to time such as a waiting time “5 minutes”.

  The work information storage unit 121 is not limited to the above, and may store various types of information according to the purpose. For example, the work information storage unit 121 may store the information when the processing of each phase is out of order. In addition, for example, the work information storage unit 121 may store the information when proceeding to the next phase when the processing of each phase that is out of order is completed. For example, in the meat potato that is the dish identified by the dish ID “CK1”, the work information storage unit 121 stores the information when the cutting work such as the second phase of cutting meat or the third phase of cutting potato is out of order. You may remember. Further, in this case, the work information storage unit 121 may store the information when proceeding to the next phase, for example, frying in a pan, when all the work to be cut off in the second phase and the third phase is completed. .

(Output information storage unit 122)
The output information storage unit 122 according to the embodiment stores various types of information related to output. FIG. 5 is a diagram illustrating an example of the output information storage unit according to the embodiment. For example, the output information storage unit 122 stores information to be output to assist the user's work. In the example of FIG. 5, the output information storage unit 122 includes items such as “cooking ID”, “phase”, “output ID”, and “content”.

  “Cooking ID” indicates identification information for identifying cooking as a work. “Phase” indicates a phase included in the dish. “Output ID” indicates identification information for identifying information to be output. “Content” indicates the content of the corresponding output.

  For example, in the example shown in FIG. 5, in the dish identified by the dish ID “CK1”, in the first phase, the contents such as “Please prepare meat, potatoes ...” identified by the output ID “OP1” are output. Indicates that information is output.

  The output information storage unit 122 is not limited to the above, and may store various types of information according to the purpose. For example, when a plurality of output information is output in each phase, the output information storage unit 122 may store a plurality of output information in association with each phase. Further, for example, the output information storage unit 122 may store the output order when the plurality of output information associated with each phase has an output order.

(Control unit 130)
Returning to the description of FIG. 3, the control unit 130 is a controller, and is stored in a storage device inside the determination apparatus 100 by, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like. Various programs (corresponding to an example of a determination program) are implemented by using the RAM as a work area. The control unit 130 is a controller, and is realized by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

  As illustrated in FIG. 3, the control unit 130 includes an acquisition unit 131, a determination unit 132, and a provision unit 133, and implements or executes information processing functions and operations such as determination processing described below. Note that the internal configuration of the control unit 130 is not limited to the configuration illustrated in FIG. 3, and may be another configuration as long as the information processing described below is performed.

(Acquisition part 131)
The acquisition unit 131 acquires various types of information. For example, the acquisition unit 131 acquires various types of information from the work information storage unit 121, the output information storage unit 122, and the like. The acquisition unit 131 may acquire various types of information from an external information processing apparatus. The acquisition unit 131 acquires various types of information from the terminal device 10 and the sensor device 20.

  For example, the acquisition unit 131 acquires image information relating to a user who performs a work including a plurality of phases, each of which includes work contents associated with each other, and in which the order between the phases is determined in advance. For example, the acquisition unit 131 acquires image information related to the user's line-of-sight direction. For example, the acquisition unit 131 acquires image information related to the user's hand. For example, the acquisition unit 131 acquires image information from a terminal device worn by the user. For example, the acquisition unit 131 acquires image information related to a user who performs cooking including a plurality of phases.

  In the example of FIG. 1, the acquisition unit 131 acquires voice information of the utterance PA11 from the terminal device 10. In the example of FIG. 1, the acquisition unit 131 acquires image information IM11 from the terminal device 10. In the example of FIG. 1, the acquisition unit 131 acquires image information IM21 from the terminal device 10.

(Determination unit 132)
The determination unit 132 determines various types of information. For example, the determination unit 132 determines a working phase that is a phase in the user's work based on the image information acquired by the acquiring unit 131 and information on a plurality of phases. For example, the determination unit 132 determines a working phase in the user's cooking based on the image information and information regarding a plurality of phases.

  In the example of FIG. 1, the determination unit 132 determines whether or not dish designation information is included in the utterance PA11. For example, the determination unit 132 determines whether the utterance PA11 includes cooking designation information by analyzing the voice information of the received utterance PA11. For example, the determination unit 132 determines whether or not the dish designation information is included in the utterance PA11 by comparing a list of dish names targeted in the dish assistance service with a character string included in the utterance PA11. In the example of FIG. 1, the determination unit 132 converts the received speech information of the utterance PA11 into character information, and indicates that the user U1 is trying to make meat potatoes in the utterance PA11 based on the converted character information. Is determined to be included.

  In the example of FIG. 1, the determination unit 132 determines the working phase of the user U1. For example, the determination unit 132 determines the working phase of the user U1 based on the determination information “DCK1-1” (see FIG. 4) regarding the first phase of meat potato that is the working phase at that time. In the example of FIG. 1, the determination unit 132 determines whether the materials are prepared by comparing the image information IM11 acquired from the terminal device 10 with the image information included in the determination information “DCK1-1”. And In the example of FIG. 1, since the image information IM11 includes meat MT1, potato PT1, and onion ON1, the determination unit 132 determines that the user U1 has completed the first phase. For example, the determination unit 132 determines that the second phase, which is the next phase after the first phase, is the working phase.

  In the example of FIG. 1, the determination unit 132 determines the working phase of the user U1. For example, the determination unit 132 determines the working phase of the user U1 based on the determination information “DCK1-2” (see FIG. 4) regarding the second phase of meat potato that is the working phase at that time.

  In the example of FIG. 1, the determination unit 132 compares the image information IM21 acquired from the terminal device 10 with the image information included in the determination information “DCK1-2”, so that the meat MT1 is cut into a bite. It shall be determined. In the example of FIG. 1, the image information IM21 includes meat MT1-1, MT1-2, and MT1-3 in which the meat MT1 is cut into a bite. Determine that the phase is complete. For example, the determination unit 132 determines that the third phase, which is the next phase after the second phase, is the working phase.

(Providing unit 133)
The providing unit 133 provides various types of information. For example, the providing unit 133 provides various information to the terminal device 10. For example, the providing unit 133 provides the output information stored in the output information storage unit 122 to the terminal device 10. In the example of FIG. 1, the providing unit 133 provides the first phase information OP1 to the terminal device 10. In the example of FIG. 1, the providing unit 133 provides the second phase information OP2 to the terminal device 10.

  In the example of FIG. 1, as shown in the work information storage unit 121-1, the providing unit 133 determines that the dish made by the user U1 is meat potato, and sets the first phase of meat potato as the working phase. For example, the providing unit 133 provides output information corresponding to the working phase to the terminal device 10. In the example of FIG. 1, the providing unit 133 has the first content “Please prepare meat, potatoes ...” corresponding to the first phase that is the working phase, as shown in the output information storage unit 122-1. The phase information OP1 is provided to the terminal device 10.

  In the example of FIG. 1, the providing unit 133 sets the second phase of meat potato as the working phase because the providing unit 133 determines that the working phase of the user U1 is the second phase, as shown in the work information storage unit 121-2. To do. For example, the providing unit 133 provides output information corresponding to the working phase to the terminal device 10. In the example of FIG. 1, the providing unit 133, as shown in the output information storage unit 122-2, corresponds to the second phase, which is the working phase, in the second phase of “Please cut the meat into bite-sized pieces”. Information OP2 is provided to the terminal device 10.

  In the example of FIG. 1, since the providing unit 133 determines that the working phase of the user U1 is the third phase as illustrated in the working information storage unit 121-3, the third phase of meat potato is set as the working phase. To do. For example, the providing unit 133 provides output information corresponding to the working phase to the terminal device 10. In the example of FIG. 1, as shown in the output information storage unit 122-3, the providing unit 133 recognizes “potatoes” identified by the output ID “OP3” that is output information corresponding to the third phase that is the working phase. The terminal device 10 is provided with output information (third phase information OP3) indicating that “please cut it into a bite”.

[3. Configuration of terminal device]
Next, the configuration of the terminal device 10 according to the embodiment will be described with reference to FIG. FIG. 6 is a diagram illustrating a configuration example of the terminal device according to the embodiment. As illustrated in FIG. 6, the terminal device 10 includes a communication unit 11, a storage unit 12, an input unit 13, a display unit 14, and a control unit 15. Moreover, the terminal device 10 may have an insertion port (an earphone jack or the like) of a terminal of an external device such as a speaker or an earphone that outputs as sound.

(Communication unit 11)
The communication unit 11 is realized by a communication circuit or the like, for example. The communication unit 11 is connected to a predetermined communication network (not shown) by wire or wireless, and transmits / receives information to / from the determination device 100.

(Storage unit 12)
The storage unit 12 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 12 stores, for example, information related to an application (for example, a cooking support application) installed in the terminal device 10, such as a program.

(Input unit 13)
The input unit 13 receives various operations from the user. For example, the input unit 13 may accept various operations from the user via a display surface (for example, the output unit 153) by a touch panel function. Further, the input unit 13 may accept various operations from buttons provided on the terminal device 10 or a keyboard or mouse connected to the terminal device 10.

(Display unit 14)
The display unit 14 is a display screen such as a tablet terminal realized by, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, and is a display device for displaying various information.

(Control unit 15)
The control unit 15 is a controller, and is realized, for example, by executing various programs stored in a storage device such as the storage unit 12 inside the terminal device 10 using the RAM as a work area by a CPU, an MPU, or the like. . For example, the various programs include an installed application program (for example, a cooking support application). The control unit 15 is a controller, and is realized by an integrated circuit such as an ASIC or FPGA, for example.

  As illustrated in FIG. 6, the control unit 15 includes a transmission unit 151, a reception unit 152, and an output unit 153, and realizes or executes functions and operations of output processing described below. Note that the internal configuration of the control unit 15 is not limited to the configuration illustrated in FIG. 6, and may be another configuration as long as it performs output processing described later.

  The transmission unit 151 transmits various information. For example, the transmission unit 151 transmits predetermined information to the external device in accordance with the input from the user received by the input unit 13. For example, the transmission unit 151 transmits predetermined information to the external device according to the user's utterance detected by a predetermined microphone or the like. For example, the transmission unit 151 transmits various types of information to the determination apparatus 100 using the activated application. In the example of FIG. 1, the transmission unit 151 transmits information regarding the utterance PA11 of the user U1 to the determination device 100. In the example of FIG. 1, the transmission unit 151 transmits the acquired voice information of the utterance PA11 to the determination apparatus 100 as input information. In the example of FIG. 1, the transmission unit 151 transmits the image information IM11 received from the sensor device 20 to the determination device 100. In the example of FIG. 1, the transmission unit 151 transmits the image information IM21 received from the sensor device 20 to the determination device 100.

  The receiving unit 152 receives various information. The receiving unit 152 receives various information from the determination device 100, the sensor device 20, and the like. In the example of FIG. 1, the reception unit 152 receives image information IM11 from the sensor device 20. In the example of FIG. 1, the reception unit 152 receives image information IM <b> 21 from the sensor device 20. In the example of FIG. 1, the reception unit 152 receives the first phase information OP <b> 1 from the determination device 100. In the example of FIG. 1, the reception unit 152 receives the second phase information OP2 from the determination device 100. In the example of FIG. 1, the reception unit 152 receives (acquires) the utterance PA11 “Meat potato is made” as an input.

  The output unit 153 outputs various information. The output unit 153 outputs the sound through a speaker or the like. Further, for example, the output unit 153 may be displayed as character information or image information by the display unit 14.

  For example, the output unit 153 outputs information regarding the working phase determined by the determination device 100. For example, the output unit 153 outputs information related to the working phase as audio information. For example, the output unit 153 outputs information on the work content in the work phase. In the example of FIG. 1, the output unit 153 outputs the first phase information OP1 “Please prepare meat, potatoes ...” as a sound. In the example of FIG. 1, the output unit 153 outputs the second phase information OP2 “Please cut the meat into a bite” as sound.

  For example, the output unit 153 outputs warning information when the user performs a work different from the work content in the working phase. For example, the output unit 153 outputs warning information when a predetermined period elapses without the transition to the next phase or the completion of the work after the determination apparatus 100 determines the in-work phase. For example, the output unit 153 outputs information that prompts the user to proceed with work. Details of these points will be described later.

  Note that the processing such as output processing by the control unit 15 described above may be realized by predetermined control information such as JavaScript (registered trademark). In addition, when the output process described above is performed by a predetermined application (for example, a cooking support application) or when the output process is performed by a dedicated application, the control unit 15 performs, for example, application control for controlling the predetermined application or the dedicated application. You may have a part.

[4. (Judgment process flow)
Next, the procedure of determination processing by the determination system 1 according to the embodiment will be described with reference to FIG. FIG. 7 is a flowchart illustrating an example of the determination process according to the embodiment.

  As shown in FIG. 7, the determination system 1 determines whether or not a dish has been designated (step S101). For example, the determination system 1 determines whether a user has specified a dish based on information related to the user's utterance acquired from the terminal device 10 that has detected the user's utterance. If it is determined that the dish is not specified (step S101: No), the determination system 1 repeats the process of step S101.

  On the other hand, when it determines with cooking having been designated (step S101: Yes), the determination system 1 acquires work information. For example, the determination system 1 acquires work information corresponding to the dish designation information among the work information stored in the work information storage unit 121 (see FIG. 4) (step S102).

  Then, the determination system 1 sets the first phase as a working phase (step S103). Then, the determination system 1 outputs information on the working phase (step S104). In the example of FIG. 1, the terminal device 10 of the determination system 1 outputs the first phase information OP1 “Please prepare meat, potatoes ...” as a voice.

  Then, the determination system 1 acquires image information (step S105). Then, the determination system 1 determines whether the work in the work phase is completed based on the acquired image information (step S106). If it is determined that the work in the work phase is not completed (step S106: No), the determination system 1 repeats the process of step S105.

  On the other hand, when it is determined that the work in the work phase is completed (step S106: Yes), the determination system 1 determines whether all the phases are completed (step S107). When it is determined that all the phases are not completed (step S107: No), the determination system 1 shifts the working phase to the next phase (step S108). Thereafter, the determination system 1 returns to step S104 and repeats the process.

  On the other hand, when it determines with all the phases having been completed (step S107: Yes), the determination system 1 complete | finishes a process.

[5. (Judgment process flow)
Next, the procedure of determination processing by the determination system 1 according to the embodiment will be described with reference to FIG. FIG. 8 is a sequence diagram illustrating an example of the determination process according to the embodiment.

  As illustrated in FIG. 8, the determination device 100 acquires dish designation information from the terminal device 10 (step S201). For example, the determination apparatus 100 acquires cooking designation information corresponding to the content of the user's utterance from the terminal device 10 that has detected the user's utterance.

  And the determination apparatus 100 acquires work information based on cooking designation information (step S202). For example, the determination apparatus 100 acquires work information corresponding to dish designation information among the work information stored in the work information storage unit 121 (see FIG. 4).

  Moreover, the determination apparatus 100 provides the first phase information among the work information to the terminal device 10 (step S203). In the example of FIG. 1, the determination device 100 provides the first phase information OP <b> 1 among the output information stored in the output information storage unit 122 (see FIG. 5) to the terminal device 10.

  The terminal device 10 that has received the first phase information from the determination device 100 outputs the first phase information (step S204). In the example of FIG. 1, the terminal device 10 outputs the first phase information OP1 “Please prepare meat, potatoes ...” as a voice.

  Then, the terminal device 10 receives image information from the sensor device 20 (step S205). In the example of FIG. 1, the terminal device 10 receives image information transmitted from a sensor device 20 that is a glasses-type imaging device worn by a user. For example, the terminal device 10 may receive the image information transmitted from the sensor device 20 by instructing the sensor device 20 to transmit the image information.

  And the terminal device 10 which received image information from the sensor apparatus 20 transmits image information to the determination apparatus 100 (step S206). In the example of FIG. 1, the terminal device 10 transmits image information IM11 including meat MT1, potato PT1, onion ON1, and the like to the determination device 100.

  The determination apparatus 100 that has acquired the image information from the terminal device 10 performs a determination process based on the image information (step S207). In the example of FIG. 1, the determination apparatus 100 determines whether the user U1 has completed the first phase based on the image information IM11 including the meat MT1, potato PT1, onion ON1, and the like. If the determination device 100 determines that the user U1 has completed the first phase, the determination device 100 transitions to the second phase next to the first phase, and provides the terminal device 10 with the second phase information OP2. If the determination device 100 determines that the user U1 has not completed the first phase, the determination device 100 provides predetermined information or waits until the next image information is acquired. Note that the determination apparatus 100 performs the same determination process in the second and subsequent phases, and thus the description thereof is omitted.

[6. Operation error output)
Note that the determination system 1 may output various types of information according to the user's work content. For example, the determination device 100 may output a predetermined warning when the user's work content is different from the content of the working phase. This point will be described with reference to FIG. FIG. 9 is a diagram illustrating an example of work error notification in the determination system according to the embodiment. In FIG. 9, the description of the same points as in FIG. 1 will be omitted as appropriate.

  The terminal device 10-3 corresponds to the terminal device 10-3 in FIG. The output storage unit 121-2 corresponds to the output storage unit 121-2 in FIG. In the example of FIG. 9, an output information storage unit 122A that includes predetermined warning information as output information is shown. Except for the above points, the output information storage unit 122A is the same as the output information storage unit 122. The example of FIG. 9 shows a case where the user's working phase is the second phase of meat potato.

  First, the example of FIG. 9 shows a state in which the onion ON1 is cut into the onions ON1-1 and ON1-2 on the cutting board CB, and the sensor device 20 transmits image information IM31 obtained by imaging the situation to the terminal device 10. . Then, the terminal device 10-3 receives the image information IM31 (Step S31). Thereafter, the terminal device 10-3 transmits the image information IM31 to the determination device 100 (step S32).

  The determination apparatus 100 that has acquired the image information IM31 from the terminal device 10-3 determines the working phase of the user U1 (step S33). For example, the determination apparatus 100 determines the working phase of the user U1 based on the determination information “DCK1-2” (see FIG. 4) regarding the second phase of meat potato that is the working phase at that time.

  For example, the determination information “DCK1-2” includes image information in a state in which the meat MT1 is formed into a bite size, and the determination device 100 determines whether the image information IM31 acquired from the terminal device 10-3 and the determination information It is determined whether or not the meat MT1 is cut into a bite by comparing image information included in the information “DCK1-2”. In the example of FIG. 9, since the image information IM31 includes onions ON1-1 and ON1-2 in which the onions ON1 that are not meat MT1 are cut into two, the determination device 100 determines that the user U1 is the second one. It is determined that the work different from the phase is being performed. That is, the determination apparatus 100 determines that the working phase remains the second phase and the second phase is not completed.

  Therefore, in the example of FIG. 9, the determination apparatus 100 determines that the in-working phase of the user U1 remains the second phase as illustrated in the work information storage unit 121-2, and therefore determines the in-working phase as the second phase. Leave. Moreover, the determination apparatus 100 determines that the work content of the user U1 is incorrect based on the above determination.

  And the determination apparatus 100 provides the output information corresponding to all the phases to the terminal device 10-3 (step S34). In the example of FIG. 9, the determination device 100 has a content of “work content seems to be different” identified by the output ID “ER1” which is output information corresponding to all phases, as shown in the output information storage unit 122A. Output information (warning information ER1) is provided to the terminal device 10-3. And the terminal device 10 which received warning information ER1 outputs warning information ER1 as an audio | voice (step S35). In the example of FIG. 9, the terminal device 10-4 outputs the warning information ER <b> 1 as sound using a speaker or an earphone worn by the user U <b> 1. The voice “The work contents seem to be different” included in the warning information ER1 enables the user U1 to recognize that his / her work is wrong, and can appropriately change the work.

[7. effect〕
As described above, the determination system 1 according to the embodiment includes the acquisition unit 131 and the determination unit 132. The acquisition unit 131 acquires image information relating to a user who performs a work including a plurality of phases, each of which includes work contents associated with each other, and in which the order between the phases is determined in advance. The determination unit 132 determines a working phase that is a phase in the user's work based on the image information acquired by the acquiring unit 131 and information on a plurality of phases.

  As a result, the determination system 1 according to the embodiment provides appropriate assistance to a user who performs work by determining a working phase that is a phase in the user's work based on the image information and information on the plurality of phases. Can be possible.

  In the determination system 1 according to the embodiment, the acquisition unit 131 acquires image information related to the user's line-of-sight direction.

  Thereby, the determination system 1 according to the embodiment can enable appropriate assistance to a user who performs work by acquiring image information related to the user's line-of-sight direction.

  In the determination system 1 according to the embodiment, the acquisition unit 131 acquires image information related to the user's hand.

  Thereby, the determination system 1 according to the embodiment can enable appropriate assistance for the user who performs the work by acquiring the image information regarding the user's hand.

  In the determination system 1 according to the embodiment, the acquisition unit 131 acquires image information from a wearable terminal worn by the user (in the embodiment, “sensor device 20”; the same applies hereinafter).

  Thereby, the determination system 1 according to the embodiment can enable appropriate assistance for the user who performs the work by acquiring the image information from the wearable terminal of the user.

  In addition, the determination system 1 according to the embodiment includes an output unit 153. The output unit 153 outputs information related to the working phase determined by the determination unit 132.

  Thereby, the determination system 1 according to the embodiment can appropriately assist the user who performs the work by outputting information on the working phase.

  Further, in the determination system 1 according to the embodiment, the output unit 153 outputs information related to the working phase as audio information.

  Accordingly, the determination system 1 according to the embodiment can appropriately assist the user who performs the work by outputting the information related to the working phase as audio information.

  Further, in the determination system 1 according to the embodiment, the output unit 153 outputs information on the work content in the working phase.

  Thereby, the determination system 1 according to the embodiment can appropriately assist the user who performs the work by outputting information on the work content in the working phase.

  In the determination system 1 according to the embodiment, the output unit 153 outputs warning information when the user performs a work different from the work content in the work phase.

  Thereby, the determination system 1 according to the embodiment enables appropriate assistance to the user who performs the work by outputting the warning information when the user performs a work different from the work content in the work phase. Can do.

  Further, in the determination system 1 according to the embodiment, the output unit 153 displays warning information when a predetermined period has elapsed without the transition to the next phase or the completion of the operation after the determination unit 132 determines the in-work phase. Output.

  Thereby, the determination system 1 according to the embodiment performs the operation by outputting the warning information when the predetermined period has elapsed without the transition to the next phase or the completion of the operation after the determination of the in-operation phase. Appropriate assistance for the user can be made possible.

  In the determination system 1 according to the embodiment, the output unit 153 outputs information that prompts the user to proceed with work.

  Thereby, the determination system 1 according to the embodiment can appropriately assist the user who performs the work by outputting information that prompts the user to proceed with the work.

  In the determination system 1 according to the embodiment, the acquisition unit 131 acquires image information related to a user who performs cooking including a plurality of phases. The determination unit 132 determines a working phase in the user's cooking based on the image information and information regarding a plurality of phases.

  Thereby, the determination system 1 which concerns on embodiment can enable suitable assistance with respect to the user who cooks.

[8. Hardware configuration)
The determination device 100 and the terminal device 10 according to the above-described embodiments are realized by a computer 1000 having a configuration as shown in FIG. 10, for example. FIG. 10 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the determination device and the terminal device. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, an HDD (Hard Disk Drive) 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F). ) 1700.

  The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400 and controls each unit. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started up, a program depending on the hardware of the computer 1000, and the like.

  The HDD 1400 stores programs executed by the CPU 1100, data used by the programs, and the like. The communication interface 1500 receives data from other devices via the network N and sends the data to the CPU 1100, and transmits data generated by the CPU 1100 to other devices via the network N.

  The CPU 1100 controls an output device such as a display and a printer and an input device such as a keyboard and a mouse via the input / output interface 1600. The CPU 1100 acquires data from the input device via the input / output interface 1600. In addition, the CPU 1100 outputs the generated data to the output device via the input / output interface 1600.

  The media interface 1700 reads a program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads such a program from the recording medium 1800 onto the RAM 1200 via the media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. Etc.

  For example, when the computer 1000 functions as the determination device 100 or the terminal device 10 according to the embodiment, the CPU 1100 of the computer 1000 executes the program loaded on the RAM 1200 to thereby execute the functions of the control unit 130 and the control unit 15. Realize. The CPU 1100 of the computer 1000 reads these programs from the recording medium 1800 and executes them. However, as another example, these programs may be acquired from other devices via the network N.

  As described above, some of the embodiments of the present application have been described in detail with reference to the drawings. It is possible to implement the present invention in other forms with improvements.

[9. Others]
In addition, among the processes described in the above embodiment, all or part of the processes described as being automatically performed can be performed manually, or the processes described as being performed manually can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedures, specific names, and information including various data and parameters shown in the document and drawings can be arbitrarily changed unless otherwise specified. For example, the various types of information illustrated in each drawing is not limited to the illustrated information.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured.

  In addition, the processes described in the embodiments described above can be appropriately combined within a range in which the process contents are not contradictory.

  In addition, the “section (module, unit)” described above can be read as “means” or “circuit”. For example, the acquisition unit can be read as acquisition means or an acquisition circuit.

DESCRIPTION OF SYMBOLS 1 Determination system 100 Determination apparatus 121 Work information storage part 122 Output information storage part 130 Control part 131 Acquisition part 132 Determination part 133 Provision part 10 Terminal device 15 Control part 151 Transmission part 152 Reception part 153 Output part 20 Sensor apparatus N Network

Claims (13)

An acquisition unit that acquires image information related to a user who performs a work including a plurality of phases in which work contents are associated with each of the phases and the order between the phases is determined in advance;
A determination unit that determines a working phase that is a phase in the user's work based on the image information acquired by the acquiring unit and information on the plurality of phases;
A determination system comprising: The acquisition unit
The determination system according to claim 1, wherein the image information related to a visual line direction of the user is acquired. The acquisition unit
The determination system according to claim 1, wherein the image information relating to the hand of the user is acquired. The acquisition unit
The determination system according to any one of claims 1 to 3, wherein the image information is acquired by a wearable terminal worn by the user. The determination system according to any one of claims 1 to 4, further comprising an output unit that outputs information on the working phase determined by the determination unit. The output unit is
The determination system according to claim 5, wherein information regarding the working phase is output as voice information. The output unit is
The determination system according to claim 5 or 6, wherein information regarding work contents in the work phase is output. The output unit is
The determination system according to any one of claims 5 to 7, wherein warning information is output when the user performs an operation different from the operation content in the in-operation phase. The output unit is
9. The warning information is output when a predetermined period has elapsed without the transition to the next phase or the completion of the work after the judgment unit has judged the in-work phase. The determination system according to item 1. The output unit is
The determination system according to any one of claims 5 to 9, wherein information prompting the user to proceed with work is output. The acquisition unit
Obtaining the image information about a user who cooks including the plurality of phases;
The determination unit
The determination system according to any one of claims 1 to 10, wherein the in-work phase of the user's dish is determined based on the image information and information on the plurality of phases. A determination method executed by a computer,
An acquisition step of acquiring image information relating to a user who performs a work including a plurality of phases in which work contents are associated with each of the phases and the order between the phases is determined in advance;
A determination step of determining a working phase that is a phase in the user's work, based on the image information obtained by the obtaining step and information on the plurality of phases;
The determination method characterized by including. An acquisition procedure for acquiring image information relating to a user who performs a work including a plurality of phases in which work contents are associated with each of the phases and the order between the phases is determined in advance;
A determination procedure for determining a working phase that is a phase in the user's work based on the image information obtained by the obtaining procedure and information on the plurality of phases;
The determination program characterized by causing a computer to execute.

Images