JP2019201396A - Information processor, information processing method, information processing program, and content distribution system - Google Patents

Abstract

To perform time division of cooking images into chapters corresponding to recipe processes.SOLUTION: An information processor for performing time division of cooking images obtained by capturing a cooking procedure according to recipe information into chapters corresponding to multiple recipe processes of the recipe information comprises: an image acquisition unit which acquires a cooking image; a read unit which reads recipe information corresponding to the cooking image; an extraction unit which extracts prescribed terms included in each recipe process of the recipe information on the basis of the read recipe information; an identification unit which identifies an object appearing in the cooking image; a comparison unit which compares a result of the identification from of the identification unit with the extracted prescribed terms; and a division unit which, on the basis of the comparison by the comparison unit, divides a scene of the identified object in the cooking image into chapters corresponding to each recipe process.SELECTED DRAWING: Figure 1

Description

  The present disclosure relates to an information processing apparatus, an information processing method, an information processing program, and a content distribution system.

  In recent years, an increasing number of people cook while watching cooking content distributed over the Internet. For example, Patent Literature 1 discloses a cooking navigation system that can cook while browsing cooking videos using the Internet.

JP 2001-67414 A

  When viewing cooking content that includes a series of cooking work demonstrations according to the recipe process, the user (viewer) must perform troublesome operations as necessary. In other words, since the demonstration recording of the cooking operation is continuous for a certain period of time, when there is a recipe process that the viewer wants to see, the viewer needs to search for the scene of the desired recipe process by sending or returning the cooking content. There is.

  For this reason, it is useful for the user if the cooking content is divided into chapters. However, if the chapter division is manually set corresponding to the recipe process after the cooking video is recorded, it is complicated and takes time.

  Therefore, in view of such circumstances, it is an object of the present disclosure to provide an information processing device, an information processing method, an information processing program, and a content distribution system that time-divide a cooking video into chapters corresponding to a recipe process. .

  In order to achieve the above object, an information processing apparatus according to the present disclosure is an information processing apparatus that time-divides a cooking video obtained by photographing cooking work according to recipe information into chapters corresponding to a plurality of recipe processes of recipe information. There are a video acquisition unit for acquiring a cooking video, a reading unit for reading recipe information corresponding to the cooking video, and an extraction unit for extracting a predetermined term included in each recipe process of the recipe information based on the read recipe information And an identification unit for identifying an object appearing in the cooking video, a comparison unit for comparing the identification result of the identification unit and the extracted predetermined term, and cooking in correspondence with each recipe step based on the comparison of the comparison unit A dividing unit that divides the scene of the identified object of the video into chapters.

  In order to achieve the above object, an information processing method according to the present disclosure is an information processing that time-divides a cooking video obtained by photographing a cooking operation according to recipe information into chapters corresponding to a plurality of recipe steps of the recipe information. A method for obtaining a cooking video, a step for reading recipe information corresponding to the cooking video, a step for extracting a predetermined term included in each recipe process of the recipe information based on the read recipe information, A step of identifying an object appearing in a cooking video, a step of comparing an object identification result with an extracted predetermined term, and a scene of the identified object of the cooking video corresponding to each recipe process based on the comparison And chapter dividing.

  In order to achieve the above object, an information processing program according to the present disclosure causes a computer to execute the above information processing method.

  In order to achieve the above object, a content distribution system according to the present disclosure includes an information processing apparatus that time-divides a cooking video into chapters corresponding to a plurality of recipe processes, and a viewer that the information processing apparatus distributes the cooking video. A content distribution system including a terminal, wherein the information processing apparatus reads a recipe information corresponding to the cooking video, and a predetermined information included in each recipe process of the recipe information based on the read recipe information. Each recipe based on the comparison of the extraction unit for extracting the term, the identification unit for identifying the object shown in the cooking video, the identification result of the identification unit, and the extracted predetermined term, and the comparison unit Distribution comprising: a dividing unit that divides a scene of the identified object of the cooking video in correspondence with the process; and a distribution unit that distributes the cooking video to the viewer terminal Delivers the cooking image of the chapter corresponding to the recipe step including a specified term that has been received from a viewer terminal.

  According to the present disclosure, a cooking video can be time-divided into chapters corresponding to a recipe process. Thereby, the user can view the cooking image corresponding to the desired recipe process.

1 is a configuration diagram of a content distribution system 1. FIG. It is a figure which shows an example of recipe information. 5 is a flowchart illustrating an example of a processing flow of the content distribution server 100. It is a figure explaining the correspondence of a cooking picture and a recipe process. It is a figure which shows the example of an image of the flame | frame of FIG. It is a figure which shows an example of the cooking image delivered. 2 is a block diagram showing a hardware configuration of a content distribution server 100. FIG.

  Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all the drawings for explaining the embodiments, common constituent elements are denoted by the same reference numerals, and repeated explanation is omitted. The following embodiments do not unduly limit the contents of the present disclosure described in the claims. In addition, all the components shown in the embodiments are not necessarily essential components of the present disclosure.

  The content distribution system according to the present embodiment is a system that time-divides cooking videos into chapters corresponding to recipe processes and distributes them as cooking contents to viewers. A viewer of content can view a cooking video corresponding to a desired recipe process.

  FIG. 1 is a configuration diagram of the content distribution system 1. With reference to FIG. 1, the outline | summary and structure of the content delivery system 1 are demonstrated.

  The content distribution system 1 includes a content distribution server 100 and a viewer terminal 200, and is communicably connected via a network NW. The network NW includes a WAN (World Area Network), a LAN (Local Area Network), and the like. The number of viewer terminals 200 is not limited to the number shown in FIG. 1, and a plurality of viewer terminals 200 may be connected to the content distribution server 100 via the network NW.

  In the kitchen studio that records the demonstration of the cooking work, the talent T, who is the cook, performs cooking according to the recipe process of the recipe information. The content distribution server 100 distributes content (cooked video) based on a series of cooking videos of the talent T taken by the camera 141 to the viewer terminal 200.

  The viewer terminal 200 is a terminal used by a viewer who views content, and is, for example, a portable terminal such as a smartphone or a tablet, or an information processing terminal such as a PC (Personal Computer). The viewer terminal 200 can be realized by installing a content distribution application on a general-purpose terminal.

(Configuration of content distribution server 100)
The content distribution server 100 corresponds to an information processing apparatus, and includes a communication unit 110, a storage unit 120, and a control unit 130.

  The communication unit 110 corresponds to a distribution unit, and is a communication interface for communicating with the viewer terminal 200 via the network NW. The communication unit 110 delivers the cooking video to the viewer terminal 200 as cooking content. Moreover, the communication part 110 may deliver the cooking image of the chapter corresponding to the recipe process including the designated term received from the viewer terminal 200.

  The storage unit 120 stores recipe information. The recipe information includes at least a recipe process indicating a procedure of cooking work. In addition, the recipe process includes the names of ingredients such as “cabbage”, “carrot”, “butter”, a term indicating cooking utensils used in cooking operations such as “knives”, “frying pans”, “pans”, “fry” , “Simmer”, “cut”, and other terms indicating cooking operations.

  Note that the recipe information may include information on ingredients and cooking utensils necessary for cooking, and a cooking image that captures cooking and the state of cooking work corresponding to the recipe process. Moreover, the recipe information memorize | stored in the memory | storage part 120 may be the recipe posted from the viewer terminal 200 via the network NW, and is the recipe which the provider who manages the content delivery system 1 registered. Also good.

  FIG. 2 is a diagram illustrating an example of recipe information. An example of recipe information will be described with reference to FIG.

  The recipe information 20 includes a cooking image 21, material information 22, and a recipe process 23. The dish image 21 is an image of the dish indicated by the recipe information 20, and is an image of a snowball cookie in the example of FIG. The material information 22 is information related to ingredients necessary for cooking indicated by the recipe information 20.

  The recipe process 23 shows the process 1-5 which is the procedure of the cooking which the recipe information 20 shows, ie, a snowball cookie. The recipe process includes the names of ingredients such as “butter” and “soft flour”, terms indicating cooking operations such as “mixing” and “arrangement”, and cooking utensils used in cooking operations such as “whather” and “rubbera” Including the terminology.

  Returning to FIG. 1, the controller 130 will be described. The control unit 130 includes a video acquisition unit 131, a reading unit 132, an extraction unit 133, an identification unit 134, a comparison unit 135, and a division unit 136.

  The video acquisition unit 131 acquires a cooking video of the cooking work according to the recipe information, which is captured by the camera 141 that is a photographing device. Specifically, an image obtained by photographing a series of cooking operations according to recipe information of the talent T who is the cook is acquired. For example, the talent T performs each recipe process until cooking is completed.

  A camera 141 is installed in the kitchen studio. The camera 141 can shoot by changing the shooting range such as the hand of the talent T and the front posture. The front posture is a posture in which the expression and movement of the talent T and the state of cooking work can be understood.

  A plurality of cameras may be installed in the kitchen studio. For example, a camera with a shooting range in the oven or microwave oven, or a camera with a stove as the shooting range may be installed, and when cooking with multiple people, each front posture and hand are set as the shooting range. A camera may be installed. And the cooking video image | photographed with the some camera may be edited into one cooking video, and you may make the video acquisition part 131 acquire. Further, the sound of the talent T and the cooking sound in the kitchen studio are collected by a microphone (not shown).

  The reading unit 132 reads recipe information from the storage unit 120. Specifically, recipe information corresponding to the cooking work performed by the talent T is read.

  The extraction unit 133 extracts predetermined terms included in each recipe process from the recipe information read by the reading unit 132. Here, the predetermined term is a term indicating the name of the food, the name of the cooking utensil used in the cooking operation, and / or the cooking operation. In addition, the extraction part 133 may extract the term which shows the name of several foodstuffs, the name of several cooking utensils, and several cooking work from a recipe process.

  The names of the cooking utensils used in the cooking operation are, for example, “knives”, “frying pans”, “pots”, “scalars”, “frothers”, etc., but cooking machines such as “microwave ovens”, “ovens” etc. The name may be included.

  Moreover, the term which shows cooking operation | movement is a term showing the operation | work which a chef performs, such as "stir fry", "boil", "steam", "cut", "mix", and "wash".

  The identification unit 134 identifies an object shown in the cooking video acquired from the video acquisition unit 131. For example, a frame of a cooking video is cut out in a predetermined unit region, and a feature amount such as an outline of an object in the unit region is extracted. Using a database (not shown) in which the feature quantity is associated with the ingredients and cooking utensils in advance, the ingredients and cooking utensils associated with the extracted feature quantities are searched. Alternatively, a template image of food or cooking utensil may be prepared in advance, and the food image or cooking utensil may be searched by comparing the template image and the frame image.

  Moreover, the identification part 134 identifies the cooking operation reflected in the cooking video. Specifically, the object shown in the cooking video is identified, and the content of the cooking operation is determined based on the identified object. For example, when “knife”, “chopping board”, “green pepper”, “hand”, etc. are identified as objects for a certain frame, it is determined that the cooking operation is “cut”. If “frying pan”, “kibera”, “cabbage”, “pork”, etc. are identified as objects, it is determined that the cooking operation is “stir fry”. That is, the cooking operation is determined by the combination of the identified objects. The combination of objects and the corresponding cooking operation may be registered in advance by a business operator that operates the content distribution system 1. In addition, the identification method of a foodstuff, a cooking utensil, and cooking work is not limited to this, A well-known image recognition technique etc. can be applied.

  The comparison unit 135 compares the identification result output from the identification unit 134 with the predetermined term extracted by the extraction unit 133 for each frame. Specifically, the object in the frame identified by the identification unit 134, the contents of the cooking utensil, the contents of the cooking operation, and the terms included in each recipe process extracted by the extraction unit 133 are compared. For example, the degree of coincidence indicating how much the identification result matches the extracted term is calculated, and it is determined which recipe process has the high degree of coincidence. Then, as a result of the comparison, the frame to be compared is associated with the recipe process having a high degree of coincidence.

  Moreover, since the cooking video is a video of cooking work according to the recipe process, the frames are arranged in the order of the recipe process along the time axis. Therefore, for example, the comparison unit 135 can determine the corresponding recipe process by comparing the recipe image in order from the first frame of the cooking video.

  Based on the comparison by the comparison unit 135, the dividing unit 136 divides the scene of the identified object of the cooking video in accordance with each recipe step. Specifically, since each frame and the recipe process are associated with each other by the comparison of the comparison unit 135, the dividing unit 136 divides the chapter at a timing when the associated recipe process is changed to the next recipe process.

  In addition to the comparison result of the comparison unit 135, the division unit 136 may also divide chapters with reference to the change or difference in the feature amount in the frame of the cooking video. In other words, the content of the cooking process in the recipe process is less similar to the image, such as “mix”, “cut”, “sauté”, etc., so the content of the frame is large when changing from one process to the next. It will change. Therefore, the dividing unit 136 extracts a frame near the time point at which the recipe process is considered to change according to the comparison result of the comparing unit 135 to obtain a change or difference in the feature amount, and the change amount or difference exceeds a predetermined threshold value. In such a case, chapter division may be performed between the two frames.

  Further, the dividing unit 136 associates the divided chapters with the recipe process associated with each frame by the comparison by the comparison unit 135 and causes the storage unit 120 to store them. At this time, a predetermined term included in each recipe process extracted by the extraction unit 133 may also be stored in association with each chapter.

(Processing flow of content distribution server 100)
FIG. 3 is a flowchart showing an example of the processing flow of the content distribution server 100.
FIG. 4 is a diagram for explaining the correspondence between the cooking video and the recipe process. FIG. 5 is a diagram illustrating an example of an image of the frame in FIG. The processing flow of the content distribution server 100 will be described with reference to FIGS. In FIG. 4, frames 31 to 35 are frame images of cooking videos, and recipe steps 1 to 5 correspond to the recipe information of FIG.

  In step S301 of FIG. 3, the video acquisition unit 131 acquires a cooking video from the camera 141 that captures the cooking work according to the recipe information.

  In step S <b> 302, the reading unit 132 reads recipe information corresponding to the cooking operation photographed by the camera 141 from the storage unit 120.

  In step S303, the extraction unit 133 extracts the names of ingredients, the names of cooking utensils used in the cooking work, and the terms indicating the cooking work included in each recipe process from the read recipe process. For example, in the recipe process 1 of FIG. 4, “bowl”, “butter”, “put”, and in the recipe process 2, “powdered sugar”, “add”, “whather”, and “mix” are extracted.

  In step S304, the identification unit 134 identifies an object from the top frame of the cooking video. For example, referring to the examples of FIGS. 4 and 5, “bowl” and “butter” are identified in the frame 31 at time t1. Similarly, in a frame (not shown) at time t (t1 <t <t2), “bowl” and “butter” are identified.

  Next, in the frame 32 at time t2, “bowl”, “butter”, “powdered sugar”, and “frother” are identified. Similarly, in a frame (not shown) at time t (t2 <t <t3), “bowl”, “butter”, “powdered sugar”, and “frother” are identified. In the frame at time t (t2 ≦ t <t3), “mix” is identified as the content of the cooking operation from the identified combination of objects.

  In the frame 33 at time t3, “rubber spatula”, “bowl”, and “person” are identified. Further, in the frame (not shown) at time t (t3 <t <t4), “dough” is also identified, and “mixing” and “rounding” are identified as the contents of the cooking operation from the combination of the identified objects. .

  In the frame 34 at time t4, “top” and “fabric” are identified. Further, in the frame (not shown) at time t (t4 <t <t5), “oven”, “person”, and the like are identified. In the frame 35 at time t5, “cookie” and “dish” are identified.

  In step S305, the comparison unit 135 compares the identification result of the identification unit 134 with the predetermined term extracted by the extraction unit 133 for each frame. 4 and 5, in the frame 31, the identification unit 134 identifies “bowl” and “butter”. The extraction unit 133 extracts “bowl”, “butter”, and “put” from the recipe process 1. The comparison unit 135 compares the object identified in order from the beginning of the recipe process with the extracted term for the frame 31 that is the first frame of the cooking video, and calculates the degree of coincidence. In this example, it is determined that the frame 31 corresponds to the recipe process 1 which is the first process. The comparison unit 135 sequentially compares the frames, and determines that the frame at time t (t1 <t <t2) corresponds to the recipe process 1.

  Similarly, the frame at time t (t2 ≦ t <t3) is in the recipe step 2, the frame at time t (t3 ≦ t <t4) is in the recipe step 3, and the frame at time t (t4 ≦ t <t5) is In the recipe process 4, the frame at time t (t5 ≦ t <until the end of the cooking video) is associated with the recipe process 5.

  In step S306, the dividing unit 136 reads the cooking video from the first frame, compares the comparison unit 135, that is, refers to the associated recipe process, and performs chapter division at the timing when the recipe process has proceeded next. In the example of FIG. 4, the frame at time t (t1 ≦ t <t2) is associated with the recipe process 1, but the recipe process 2 is associated with the frame 32 at time t2. Therefore, the frame 32 in which the associated recipe process is switched is divided into chapters as the first chapter. At this time, the switching of the recipe process may be determined with reference to the feature amount of the frame and the like, and the chapter division may be performed. Moreover, words (for example, “next” etc.) uttered by the cook may be detected by voice recognition to determine the switching of the recipe process.

  Note that the identification unit 134 and the comparison unit 135 may identify and compare frames at predetermined intervals. This is because the cooking video contains a series of cooking operations, and the occurrence of an error in the location where the recipe process is switched (frame) does not become a fatal problem. This is because it is considered that it is sufficiently useful if it can be classified for each. Thereby, the processing amount in the control part 130 can be reduced.

  The dividing unit 136 causes the storage unit 120 to store the chapter-divided cooking video. At this time, a predetermined term included in each recipe process extracted by the extraction unit 133 is also stored in association with each chapter.

  In step S <b> 307, the communication unit 110 distributes the chapter-divided cooking video to the viewer terminal 200. Thereby, in the viewer terminal 200, the viewer can view from a chapter corresponding to a desired recipe process.

  Moreover, the communication part 110 may distribute only the cooking video of the chapter designated from the viewer terminal 200 instead of distributing the whole cooking video. Moreover, you may make it the communication part 110 deliver the chapter image | video about the recipe process containing the designated term received from the viewer terminal 200. FIG. As a result, the viewer does not browse the recipe process and select / specify the cooking process that the viewer considers difficult, but simply designates a term indicating the content he / she wants to view, and the recipe process is performed for that part. Can watch. Thereby, the trouble of the viewer who browses and designates all recipe processes can be reduced.

  FIG. 6 is a diagram illustrating an example of a cooking video to be distributed. The corresponding recipe process 62 is highlighted on the screen 61 of the viewer terminal 200 together with the cooking video. A recipe process 63 that is the next process of the recipe process 62 is also displayed. Thereby, the viewer can view the cooking video while being aware of the next recipe process. In addition, the viewer can display the chapter video corresponding to the recipe process 63 on the screen 61 by selecting and instructing the recipe process 63 with, for example, a tap. Thereby, the viewer can view the video of the recipe process 63 immediately.

  Alternatively, all recipe processes may be displayed in chronological order and highlighted so that the cooking video currently displayed on the screen can be identified. As a result, the viewer can grasp at which time position the currently displayed recipe process is located in the entire recipe process.

(Hardware configuration diagram)
FIG. 7 is a block diagram illustrating a hardware configuration of the content distribution server 100. The content distribution server 100 is mounted on the computer 701. The computer 701 includes a CPU 702, a main storage device 703, an auxiliary storage device 704, and an interface 705.

  The operation of each component of the content distribution server 100 is stored in the auxiliary storage device 704 in the form of a program. The CPU 702 reads the program from the auxiliary storage device 704, develops it in the main storage device 703, and executes the above processing according to the program. Further, the CPU 702 secures a storage area in the main storage device 703 according to the program. Specifically, the program is a content distribution program for distributing a cooking video time-divided into chapters corresponding to the recipe process to the computer 701.

  The auxiliary storage device 704 is an example of a tangible medium that is not temporary. Other examples of the tangible medium that is not temporary include a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, and a semiconductor memory connected via the interface 705. When this program is distributed to the computer 701 via a network, the computer 701 that has received the distribution may develop the program in the main storage device 703 and execute the processing.

  The program may be for realizing a part of the functions described above. Further, the program may be a so-called difference file (difference program) that realizes the above-described function in combination with another program already stored in the auxiliary storage device 704. Note that the hardware configuration shown in FIG. 7 may be the same for the viewer terminal 200. The operation of each component of the viewer terminal 200 is also realized by a CPU according to a program stored in the auxiliary storage device, similar to the content distribution server 100 described above.

(Explanation of effect)
In the content distribution system according to the present embodiment, the cooking video is time-divided into chapters corresponding to the recipe process. Thereby, the user (viewer) can view the cooking video corresponding to the desired recipe process.

  Further, in the content distribution system according to the present embodiment, the correspondence between the recipe process is based on a comparison between the terms indicating ingredients, cooking utensils, and cooking work content included in the recipe process and the identified object in the frame. Since it determines, a cooking image | video and a recipe process can be matched easily.

  In addition, the content distribution system distributes chapter videos for the recipe process including the specified term received from the viewer terminal. As a result, for the video that the user wants to watch, the user does not browse the recipe process and select / designate the recipe process, but simply specifies a term indicating the contents of the cooking work or the like to be viewed. You can watch the process. Thereby, the trouble of the user who browses all the recipe processes and designates the recipe process can be reduced.

  The present embodiment can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the invention described in the claims and equivalents thereof, as long as they are included in the scope and gist of the invention.

1 content distribution system, 100 content distribution server, 110 communication unit, 120 storage unit, 130 control unit, 131 video acquisition unit, 132 reading unit, 133 extraction unit, 134 identification unit, 135 comparison unit, 136 dividing unit, 141 camera, 701 Computer, 702 CPU, 703 Main storage device, 704 Auxiliary storage device, 705 interface

Claims (9)

An information processing apparatus that time-divides a cooking video obtained by photographing cooking work according to recipe information into chapters corresponding to a plurality of recipe steps of the recipe information,
A video acquisition unit for acquiring the cooking video;
A reading unit for reading the recipe information corresponding to the cooking video;
Based on the read recipe information, an extraction unit that extracts predetermined terms included in each recipe process of the recipe information;
An identification unit for identifying an object shown in the cooking image;
A comparison unit that compares the identification result of the identification unit with the extracted predetermined term;
An information processing apparatus comprising: a division unit that divides a scene of the identified object of the cooking video in accordance with each recipe process based on a comparison of the comparison unit.   The information processing apparatus according to claim 1, wherein the predetermined term and the object are food ingredients.   The information processing apparatus according to claim 1, wherein the predetermined term and the object are cooking utensils.   The information processing apparatus according to claim 1, wherein the predetermined terms and objects are food materials and cooking utensils.   The predetermined term is a term indicating the cooking operation, and the identification unit determines, as the identification result, the content of the cooking operation reflected in the cooking image, which is determined based on the identification of the object reflected in the cooking image. The information processing apparatus according to claim 1, wherein the information processing apparatus outputs the information to the comparison unit. A delivery unit for delivering the cooking video to a viewer terminal;
The information processing apparatus according to claim 1, wherein the distribution unit distributes the cooking image of a chapter corresponding to a recipe process including a specified term received from the viewer terminal. An information processing method for time-dividing a cooking video obtained by photographing cooking work according to recipe information into chapters corresponding to a plurality of recipe steps of the recipe information,
Obtaining the cooking video;
Reading the recipe information corresponding to the cooking video;
Extracting predetermined terms included in each recipe process of the recipe information based on the read recipe information;
Identifying an object shown in the cooking video;
Comparing the identification result of the object with the extracted predetermined term;
An information processing method comprising: dividing the scene of the identified object of the cooking video into chapters in correspondence with the recipe steps based on the comparison.   A program for causing a computer to execute the information processing method according to claim 7. An information processing apparatus that time-divides cooking video into chapters corresponding to a plurality of recipe processes, and a viewer terminal that distributes the cooking video by the information processing apparatus,
The information processing apparatus includes:
A reading unit for reading recipe information corresponding to the cooking video;
Based on the read recipe information, an extraction unit that extracts predetermined terms included in each recipe process of the recipe information;
An identification unit for identifying an object shown in the cooking image;
A comparison unit that compares the identification result of the identification unit with the extracted predetermined term;
Based on the comparison of the comparison unit, a division unit that divides the scene of the identified object of the cooking video in correspondence with each recipe step, and
A distribution unit that distributes the cooking video to the viewer terminal, wherein the distribution unit distributes the cooking video of a chapter corresponding to a recipe process that includes a designated term received from the viewer terminal. .

Images