JP7284516B2 - Program creation support system, its method, and its program - Google Patents

Description

The present invention relates to a programming support system, its method, and its program, and more particularly to programming support technology suitable for programming learning.

With the development of AI (artificial intelligence) and information technology (IT), a shortage of IT engineers is expected, and the importance of programming human resources is being emphasized. In Japan, the Ministry of Education, Culture, Sports, Science and Technology has advocated the importance of programming education for children as one of the measures to develop programming human resources, and from 2020 programming education has become mandatory in elementary schools. Various programming education systems, educational jigs, toys, etc. for children have already been proposed, and some of them have already been put to practical use.

As one of them, programming education support for beginners using computers and the Internet has been advocated. For example, in Patent Document 1, a client for a learner, a client for a learning manager, and a learning management server are connected through a network, and the process of answering a task by a learner is grasped online in real time and made into a database, Disclosed is a programming education support system that reproduces the answering process on an arbitrary time axis based on recorded data and visualizes it with a graph, so that each learner's programming proficiency can be easily and accurately grasped. .

Also, in Non-Patent Document 1, as a service for children called "Programming" developed by the Ministry of Education, Culture, Sports, Science and Technology, a program is assembled by combining multiple pictures (blocks) while operating a mouse on the screen of the display device. , discloses computer-based programming for children. Furthermore, Non-Patent Document 2 discloses a technique of creating a program using image recognition using an AR marker and operating a robot. This is supposed to automatically transfer the program to the robot by taking a picture of the alternately arranged “dosa card” and “joken card” on which the image that will be the AR marker is written with a tablet.

Japanese Patent Application Laid-Open No. 2006-227218

Ministry of Education, Culture, Sports, Science and Technology "Programin" October 22, 2013 http://www.mext.go.jp/programin/ Toshiyuki Haramaki, ``Study on the introduction of augmented reality into programming experience in science experiment classes for children,'' IPSJ 75th (2013) National Conference Proceedings (4), Information Processing Society of Japan, March 6, 2013, pp. 385-386

The technique described in Patent Document 1 is aimed at allowing the learning management server to easily and accurately grasp the programming proficiency of the learner. It has not been.

Also, since the programming introduced in Non-Patent Document 1 is operated on the screen, it is premised on the use of a personal computer (PC) or the like. Learning programming in a school class requires a considerable amount of money because PCs corresponding to the number of students are required, and students also need to learn basic operations for operating the screen of the PC. In addition, Non-Patent Document 2 discloses the concept of programming by photographing a card with an AR marker written on it with a tablet. No suggestion.
Therefore, there is a demand for a system or teaching materials that do not require much cost and that allow children to learn programming in a simple manner.

SUMMARY OF THE INVENTION An object of the present invention is to provide a program creation support system, method, and program for creating a program by a simple method.

The program creation support system according to the present invention is preferably a program creation support system that uses a computer to support program creation,
a plurality of items each having a unique code;
a table for managing the association between the codes of the plurality of items and program instructions;
Acquisition means for acquiring the code of each of the plurality of arranged goods when creating a program;
a code recognition processing unit that recognizes the code of each of the plurality of articles acquired by the acquisition means;
a program creation processing unit that refers to the table to identify instructions corresponding to the code recognized by the code recognition processing unit, and generates a program composed of the identified instructions. It is configured as a program creation support system.
The present invention can also be grasped as a program creation support method and a program creation support program realized by the above program creation support system.

According to the present invention, a program can be created by a simple method.

1 is a diagram showing the configuration of a programming support system according to Example 1; FIG. It is a figure which shows the structural example of a portable terminal. It is a figure which shows the example of a program sheet. FIG. 4 is a diagram showing an example of a chip used for creating a program; FIG. FIG. 4 is a diagram showing an example of program instructions; FIG. 4 is a diagram showing an example of program instructions; FIG. 10 is a diagram showing an example of arrangement of chips in creating a program; FIG. 10 is a diagram showing an example of arrangement of chips in creating a program; FIG. 10 is a diagram showing an example of arrangement of chips in creating a program; It is a figure which shows the structural example of a curriculum management table. FIG. 4 is a diagram showing a configuration example of a user management table; FIG. 10 is a diagram showing a configuration example of a teacher management table; It is a figure which shows the structural example of a program management table. 4 is a flow chart showing a schematic operation of creating a program; 4 is a flowchart showing processing operations for creating a program; 4 is a flowchart showing processing operations for creating a program; It is a figure which shows the example of a screen display of a portable terminal. It is a figure which shows the connection structure of a program creation assistance system. 11 is a flow chart showing a schematic operation of creating a program according to Example 3; FIG. 12 is a diagram showing an example of a task sheet used in Example 7; FIG. 12 is a diagram showing an example of a program sheet used in Example 7; FIG. 12 is a diagram showing an example of a chip used in Example 8; FIG. 21 is a diagram showing an example of arrangement of chips in Example 8; FIG. 21 is a diagram showing an example of arrangement of chips in Example 8;

A preferred example of the present invention is realized by a program creation support system using a computer. A plurality of chips having patterns on which commands are defined in advance and a predetermined sheet on which the chips are arranged are prepared. A user selectively arranges a plurality of chips on a sheet according to a purpose. This array of chips is photographed with a camera provided in a mobile terminal such as a smartphone. A processing device (e.g., a server) in a program creation support system recognizes an image of a design of each chip from an image of an array of chips acquired by a camera, identifies an instruction code that constitutes a program, and executes a program from the instruction code array. Create data. The created program data is transmitted to the portable terminal and executed. Execution of the program is displayed, for example, as a moving image of a character on the screen of the display unit of the mobile terminal.

According to a preferred example, in a class at a school where the users are a large number of students, a program creation class can be conducted with a single portable terminal possessed by the teacher. That is, every time each student finishes arranging the chips, the arrangement of the chips is photographed by the camera of the portable terminal, the image is transmitted to the server, and the server creates a program. Each execution of the program can be viewed as a moving image on the screen of the portable terminal, so that each student can check the performance of the program.

According to a preferred example of the present invention, a user can create a program by a simple task of arranging chips on a sheet, compared to the program creation work of creating a program by a single user while operating the screen of a PC. can. Therefore, it is easy to learn the procedure for creating a program, and the burden on both the teacher and the student who teach the creation of the program is small. In addition, since it is not necessary to prepare a PC for each user, it is suitable for situations in which a large number of people study at the same time. In addition, since it is possible to prepare a chip and sheet set for each user, it can be realized at a low cost.

An embodiment of the present invention will be described in detail below with reference to the drawings.
FIG. 1 shows an example of the overall configuration of a program creation support system.
The program creation support system is configured by connecting a server 1 that assists program creation and a plurality of mobile terminals 2 used by users via a communication network 9 .

The server 1 is a computer that performs various data processing such as managing user information, communicating data with portable terminals, creating and executing programs, and the like. The details of the function and processing will be described later. Although the hardware configuration of the server 1 is not shown, it has hardware resources such as a processing unit (processor) for executing programs, memory, an external storage device such as a hard disk, an input device, and a display device.

In order to support program creation, the server 1 includes an information management unit 102 that registers and manages information on users and workers of the mobile terminal 2, a program creation processing unit 103 that creates programs, and a It has an image processing unit 104 that recognizes the image of the chip. Further, as a database (DB), it has a management information DB 105 for registering user information and various management information regarding use of programming learning, and a program DB 106 for storing information regarding program creation. The configuration and functions of these processing units and DB will be described in detail later. In the illustrated example, the management information DB 105 and the program DB 106 are configured as separate DBs, but they may be configured as one DB.

The server 1 has a site for program creation support, and by connecting to this site from the portable terminal 2, program creation support can be received.
The mobile terminal 2 is, for example, a terminal such as a smart phone (hereinafter simply referred to as a smart phone). A hardware configuration of the mobile terminal 2 will be described later with reference to FIG.

FIG. 2 shows the hardware configuration of the mobile terminal 2. As shown in FIG.
The mobile terminal 2 is, for example, a mobile terminal such as a smartphone (hereinafter simply referred to as a smartphone), and includes a processor (CPU) 202 that executes various application programs and the like, a memory 203 that stores data and programs, and a screen touch type. An input unit 204, a display unit 205 that displays a screen, a camera 206 that has an imaging function, a barcode reader 207 that reads two-dimensional barcodes, and a network connection unit 208 that performs data communication with the server 1 through the network 9. have. The smartphone can connect to many sites through the Internet, and the CPU 202 executes applications desired by the user. In addition to the above, smartphones have various functions including, for example, a voice input/output function.

[Program Sheet and Chip]
FIG. 3 shows an example of a program sheet.
A program sheet (hereinafter simply referred to as sheet) 30 is, for example, an A4 size sheet used for creating a program. In school classes, a teacher distributes a pre-printed sheet 30 to each student. The sheet 30 is preferably made of plastic rather than plain paper so that it can be used multiple times by the student and can withstand chip placement.

A program area 302 is secured in the wide space in the center of the sheet 30 . A title 301 is printed on the upper edge of the sheet 30, and an identification portion 305 including a two-dimensional bar code is printed on the upper right edge. A title 301 is, for example, the name of a curriculum. The two-dimensional barcode of the identification section 305 contains identification information for identifying the curriculum.

In the program area 302, a plurality of vertical and horizontal lines are printed to form a matrix of ruled lines, a "start" mark 303 is printed at the upper left corner of the ruled line, and a thick vertical line guideline 304 is printed downward from there. printed. Matrix lines and guidelines 304 serve as guides for arranging the chips. Vertical lines and horizontal lines in a matrix form X/Y coordinates in image processing of the chip pattern. The original information of the sheet 30 is stored in the program DB 106 as an image.

A student who is an operator arranges chips from top to bottom along the guideline 304 . Other times, chips are laid out from left to right along the matrix lines. FIGS. 7A to 7C show examples in which a plurality of chips 40 are arranged on the sheet 30. FIG.

FIG. 4 shows an example of a chip used for programming.
Chip 40 is an indicator in which program instructions are defined. For example, it is a rectangular piece of plastic about 2 cm long and 5 cm wide, which expresses (prints) the action of an instruction in a pattern. Each pattern differs for each command. It is preferable that the chip 40 has a simple adhesive material formed on its back surface so that it can be easily fixed on the sheet 30 and easily peeled off. The original information of the pattern of the chip 40 is stored in the program DB 106 as an image.

An example of 17 types of chips is shown in FIG. 4, but more chips can be added depending on the type of instruction. A plurality of chips 40 for distribution will be referred to as a chipset. The sheet 30 and the chipset 40 are sold and shipped to users in response to requests from the users. Note that the server 1 may print the sheet 30 or the chipset 40 on a designated printer in response to a request from the mobile terminal 2 .

[Example of program instruction]
FIG. 5 shows an example of program instructions (instruction list).
The program instructions consist of 9 instructions (motion instructions) representing display/movement, 2 instructions (control instructions) representing control statements, 4 instructions (input/output instructions) representing input/output, and parameters 1 to 9. It consists of an instruction (parameter) representing a numerical value up to and including four other instructions.

Each chip and command uniquely correspond, and the relationship between the chip pattern, command name, command operation, and parameter is as shown in the drawing. Remarks represent informative explanations. For example, in the action command, the picture pointing to the right (the first "move: with direction" command in Fig. 5) is a command representing "move in the specified direction", and when this arrow is pointing to the right means move right, move left when facing left. The parameters represent the number of motion commands, directions, numbers, and the like. In relation to instructions, parameters can be set for instructions for which parameters can be specified. For example, the command "move: with direction" can set the number of parameters, but the command "pick up" cannot.

As exemplified in FIG. 6, a right-facing picture "Move: Directed" command with a parameter of "3" means "move right three times". Here, the arrangement of one chip 401 representing an instruction is essential, and the arrangement of parameters 402 is arbitrary. The parameters 402 change according to the type of instruction and its execution level.

In the "other" commands, the "START" command is not used when "START" is printed on the sheet 30. FIG. The "END" instruction is placed at the end of the chip array. Instructions for operators and conditional statements are provided, but they may not be used depending on the content of the curriculum.

The relationship between the picture of the chip (that is, the image of the picture), the name indicating the action, etc., and the action of these commands is defined in a command table (denoted by reference numeral 50), which is a correspondence table, and stored in the program DB 106. . Here, an example of the screen display of the command table 50 is shown in FIG. 5. The command table 50 stored in the program DB 106 includes unique code data (command code) corresponding to the pattern image.

In one example, a plurality of instructions (referred to as an instruction set) are selected from this instruction table 50 according to the curriculum. Further, when a new chip design or a new command is defined, or when a predetermined design is desired to be deleted or changed, the contents of the command table 50 can be changed. It is possible to change the correspondence relationship between the chip pattern and the command in the command table 50 according to the administrator's input instruction from the input device (not shown) of the server 1 or the portable terminal 2 .

[Description of various tables]
Next, with reference to FIGS. 8 to 11, configurations of various tables used for supporting program creation will be described. Each table 80 to 110 is stored in management information DB 105 .

FIG. 8 shows a configuration example of a curriculum management table.
The curriculum management table 80 is a table for managing a program-created curriculum to be provided to the user and instructions used in the curriculum. Here, the curriculum may be said to be a task or problem for the user to program. Curriculum, specific task contents corresponding to the curriculum, and sheets used in the curriculum are associated with each other. This will be described later with reference to FIG.

The curriculum management table 80 includes a curriculum number for identifying a curriculum, an address for executing an application according to the curriculum, a character set representing characters used in the application, and a character set selected from the command table 50 to be used in the curriculum. Reference numbers of all instructions (reference numbers of the instruction table 50) are registered. Here, the application is a program that performs a series of processes, starting from curriculum selection, capturing images of sheets and chip arrays by the mobile terminal 2, and creating a program. A character is a character that is used as a moving image in executing a program, such as an image of a rabbit or a turtle, or an image of a famous animation.

Also, the reason why the reference numbers of all the commands used in the curriculum are registered in the command set is to speed up the program creation process. That is, in the program creation process, the process of recognizing the image of each chip from the image of the chip array is performed while referring to the instruction table 50. At this time, scanning all the chip images registered in the instruction table 50 would take a lot of time. It takes a long time. On the other hand, if the command corresponding to the reference number registered in the command set is scanned, the time required to recognize the image is greatly shortened.

FIG. 9 shows a configuration example of a user management table.
The user management table 90 is a table for managing information on users who create programs. The user management table 90 contains identification information that is a user identification ID, user name, program data storage destination index (ID) indicating an address where program data is stored, and curriculum identification information as curriculum management table 80. Curriculum number to be selected and used from, update date and time, and personal information such as the e-mail address of the mobile terminal owned by the user are registered. Curriculum numbers used in the past are sequentially registered as a log in the curriculum identification information. In this example, the program data storage location ID represents the address of the program management table 110 that manages the created program data or information designating the program management table 110 .

The teacher management table 100 is mainly used when performing program learning in a class at school or in a group because data communication is performed with the server 1 using the portable terminal 2 possessed by the teacher. Therefore, it is not necessary for the user to register information on students, including IDs for identifying students, in the user management table 90 . On the other hand, if all students have portable terminals 2 and each student independently creates a program, the user information of all students is registered in the user management table 90 .

FIG. 10 shows a configuration example of a teacher management table.
The teacher management table 100 is a table for managing information about the teacher and the students who make up the class when program learning is performed in a school class. The teacher management table 100 is selected from the curriculum management table as the teacher ID for identifying the teacher, the name of the teacher, the program data storage destination index (ID) indicating the address where the program data is stored, and the curriculum identification information. A curriculum number to be used as a teacher, a student identification ID of a class as user identification information, and an e-mail address of a mobile terminal owned by the teacher as personal information are registered. All of the student IDs that make up the class are registered in the user identification information. Curriculum numbers used in the past are sequentially registered as a log in the curriculum identification information.

FIG. 11 shows a configuration example of a program management table.
The program management table 110 is a table for managing information on created programs. The program management table 110 registers the user identification ID that created the program, the curriculum number, the created program data, the registration date and time of the program, and the modification date and time. In this embodiment, the student identification ID is registered as the user identification ID. The program data is the created program itself, and in the case of a program step of multiple lines, the number of lines is registered.

[Overview of Program Creation]
Next, with reference to FIG. 12, a schematic operation of creating a program will be described.
This example shows a flow when a teacher who has a mobile terminal 2 gives a programming class to a plurality of students. In the figure, steps indicated by solid lines represent processing operations by the mobile terminal or the server, and steps indicated by dotted lines represent actions of a human (teacher or student).

First, the teacher operates the mobile terminal 2 to enter an account and password to log in (S1201). Then, the portable terminal 2 is connected to the server 1 via the network 9, and is connected to the site for creating the program in question. Then, a curriculum selection screen is displayed on the display unit 205 of the mobile terminal 2 . The teacher selects the desired curriculum from the displayed screen (S1202).

Here, the operation from selection of the curriculum to preparation of the sheet 30 will be described. When the curriculum is selected, a curriculum list screen is displayed on the display unit 205 of the mobile terminal 2 as shown in FIG. 15(A). The teacher selects a desired curriculum (task) from the displayed curriculum list. Then, as shown in FIG. 15B, a screen showing details of the selected assignment is displayed. The teacher confirms the details of the assignment and, if he or she agrees, selects "Sheet" at the bottom of the screen. Then, a sheet corresponding to the curriculum is printed as shown in FIG. 15(C). Sheet printing can be performed using a predesignated printer. In the case of programming in a school class, it is necessary to understand the content of the class in advance, so the operations up to this point can be completed in advance and the sheet can be prepared in advance.
15A to 15C, screen information is stored in the management information DB 105, and is transmitted to and displayed on the mobile terminal 2 in response to an input instruction on the mobile terminal 2. FIG.

In an actual programming class, the teacher distributes prepared sheets 30 and chip sets 40 to a plurality of students (S1203), and sets programming questions (S1204). The assignment is to explain the assignment shown in FIG. 15(B). For example, it is a subject such as "Put an apple on the table and leave the room" corresponding to "Training 1" in the curriculum. As a slightly advanced question (for example, "Introductory 3"), there may be a question such as "Athletic meet where Mr. Rabbit and Mr. Turtle compete" without mentioning each action.

The student performs programming, that is, the task of arranging the chips 40 on the distributed sheet 30 (S1211). As a result of the arranging operation of the chips 40, the chips are arranged on the sheet 30 as shown in FIGS. 7A to 7C, for example. According to the experimental example, FIGS. 7A to 7C show an example in which the task "Let's put the apple on the table and leave the room" was answered using a plurality of different chip arrangements.

When the work of arranging the chips 40 on the sheet 30 is completed, the teacher takes an image of the chip arrangement sheet 40 that has been worked on with the camera 206 of the mobile terminal 2, and transmits the taken image to the server 1 (S1205).

The server 1 performs program creation processing based on the received image (S1206). This process will be detailed later. The program created by the server 1 is executed by the portable terminal 2 (S1207).

[Program creation process]
Next, program creation processing will be described with reference to FIGS. 13 and 14. FIG.
FIG. 13 shows a flowchart of the entire program creation process.
Prior to this process, the site managed by the server 1 is accessed from the teacher's mobile terminal 2, and personal information including the name of the user (student) is registered in the user management table 90. FIG. In the teacher management table 100, the name of the teacher, the e-mail address of the mobile terminal 2, and the identification ID of the student (user) in the class involved in the programming class are registered.

12, a curriculum is selected using the curriculum selection screen (FIG. 15), and the curriculum number selected from the curriculum management table 80 is registered in the teacher management table 100. FIG. In the processing of this example, since a different curriculum is not selected for each individual student, there is no need to register the selected curriculum number in the user management table 90 .
Also, as shown in FIG. 12, a sheet 30 and a chip set corresponding to the selected curriculum number are distributed to a plurality of students, and the students complete the task of arranging chips 40 on the sheet 30 in the programming class. and

Here, an example of chip arrangement will be described with reference to FIGS. 7A to 7C.
This example of chip arrangement is an example in which three students arranged different chips for the task "Let's put an apple on the table and leave the room." All of FIGS. 7A to 7C perform the same operation and are correct. By the way, the operation according to the program instructions based on the illustrated chip arrangement is as follows.
・Two steps to the right ・Three steps down ・Pick up ・Two steps to the left ・Place ・Two steps down ・Open the door.

Return the description to FIG. The teacher logs in by operating the portable terminal 2 (S1301). In this case, the information management unit 102 of the server 1 refers to the teacher management table 100 to confirm the teacher identification ID. A screen prompting selection is displayed, and the above processing is performed from the screen based on FIG. Then, the information management unit 102 refers to the curriculum management table 80 based on the curriculum number registered in the teacher management table 100, and activates the application registered in the curriculum management table 80 (S1302). When the application is started, the information of the curriculum management table 80 in which the selected curriculum number is registered is taken into the memory of the mobile terminal 2 and stored.

The teacher uses the barcode reader 207 of the portable terminal 2 to read the identification portion 305 of the sheet 2 . Further, the camera 206 is used to photograph the sheet (chip array sheet) 30 on which the chips 40 are arrayed. The information of the identification unit 305 and the captured image of the chip array sheet (chip array sheet image) are temporarily stored in the memory 203 (S1303). When registering a chip array sheet image in the memory 203, the teacher inputs a user identification ID (in this case, a student identification ID) from the input unit 204 in order to indicate that the image was created by a specific user (student). input. A user identification ID is assigned to the chip array sheet image and stored in the memory 203 .

Information from the identification unit 305 includes information for identifying the curriculum, and the CPU 202 compares the identification information with the curriculum number registered in the curriculum management table 80 . As a result of the comparison, if the two match, it is determined that the used sheet 30 is correct, and the subsequent processing is continued. On the other hand, if the result of the comparison is determined to be incorrect, the CPU 202 displays a message to that effect on the display unit 205 and prompts confirmation of the sheet 30 and the curriculum number.

When the regular sheet 30 corresponding to the curriculum number is used, the CPU 202 transmits the chip arrangement sheet image to the server 1 via the network connection section 208 (S1303). The transmission of the chip array sheet image to the server 1 may be performed each time a chip array sheet image for each student is obtained, or may be performed when the chip array sheet images of a plurality of students are accumulated in the memory 203. . A teacher identification ID, a curriculum number, and a user identification ID (in this example, a student identification ID) are added to the chip array sheet image and transmitted to the server 1 .

When the server 1 receives the chip arrangement sheet image, the information management section 102 confirms whether the sender's address is the mail address registered in the teacher's management table 100 . Furthermore, the student identification ID and the teacher identification ID given to the received chip arrangement sheet image are obtained, and it is confirmed whether they are registered in the teacher management table 100 . If the result of this confirmation is correct, the process moves to image processing and program creation processing. (In addition, when the individual user transmits the chip arrangement sheet image to the server from his/her mobile terminal, the information management unit 102 refers to the user management table 90 to check whether the mail address is properly registered. become.)
In the program creation process, first, the image processing unit 104 performs a process of recognizing the image of the pattern of the chip from the obtained image (S1304). Thereafter, the program creation processing unit 103 performs program creation processing based on the recognition result (S1305). These processes will be described later with reference to FIG.

The program data created by the program creation processing unit 103 is stored in the program management table 110 according to the processing of the information management unit 102 . That is, in the program management table 110, the "user ID" is the student ID of the user who created the program, the "curriculum number" is the curriculum number registered in the teacher management table 100, and the "program" is the created program. The date and time when the stored program data is registered, and the date and time when the program is stored are registered in the "program registration date and time".

After that, the program registered in the program management table 110 can be executed (S1307). In one example, program execution includes a program test and verification process before it. This is because, in the arrangement of the chips 40, there are cases where the control statement chips and parameter chips are not arranged according to a predetermined rule, or there is no "END", and these must be regarded as errors in the execution of the program. is. When such errors are found in the testing and verification process, the program data is corrected.

That is, for testing and confirmation of the program, the program data stored in the program management table 110 is transmitted via the network 9 to the mobile terminal 2 as the transmission source and executed by the CPU 202 . Execution of the program by the CPU 202 is displayed on the display unit 205 as, for example, animation of characters registered in the curriculum management table 80 . The teacher looks at the moving image displayed on the display unit 205 and judges whether the displayed moving image is correct based on the initially set "task" and the chip arrangement in the chip array sheet image acquired by the camera. As a result, when it is determined that the program is not correct, the student who created the program is instructed orally to correct the programming. At the same time, when an instruction to modify the program is given from the input unit 204, the instruction is transmitted to the server 1, and a flag (not shown) indicating that the program is being modified is stored in the program management table 110. FIG. After the student has finished correcting the programming, the corrected chip array sheet image is sent to the server 1 in the same manner as the operation described above. In this case, corrected (corrected version) program data is similarly stored in the program management table 110 . At the same time, the modification date and time of the program management table 110 are registered.

The modified program is executed on the mobile terminal 2 owned by the teacher in the same manner as described above. As a result, if there is no error, a message to that effect is sent to the server 1 and a flag (not shown) indicating completion of the program is registered in the program management table 110 .
An example of execution of the completed program will be described later with reference to FIG.

Next, the image processing (S1304) and program creation processing (S1304) will be described in detail with reference to the flowchart of FIG.
The image processing (S1304) is a process of recognizing and cutting out images of patterns drawn on individual chips 40 from the image of the chip arrangement sheet. Since the image captured by the camera 206 may be tilted, first, the tilt of the image is corrected based on the image of the sheet 30 in image correction processing (S1401). For example, if it is determined that the guideline 304 of the sheet 30 is tilted at an angle θ with respect to the vertical axis (Y-axis), the entire acquired image is rotated by an angle θ to align the vertical axis and the guideline 304 .

Next, in image pattern matching (S1402), referring to the instruction table 50, it is determined whether or not the pattern image of each chip 40 constituting the chip array matches the pattern registered in the instruction table 50. FIG. Through this processing, the code data corresponding to the image of the pattern registered in the command table 50 is selected. Even if there is an eraser, dust, etc. on the sheet 30, those images are not selected (excluded). In the image pattern matching, the obtained chip array image is not compared with all the images registered in the instruction table 50, and only the chip images corresponding to the instruction reference numbers initially registered in the curriculum management table 80 are compared. We are trying to speed up the processing time by comparing the target. Coordinate information (X/Y coordinate information on the sheet 30) of the location where the pattern image is recognized is added to the code data (instruction code) output as a result of image pattern matching.

In the program creation process (S1304), the instruction codes corresponding to the image of the pattern specified by the image processing are aligned (S1403). The reason for arranging the instruction code is that although the image of the pattern recognized by pattern matching is converted into the instruction code, the arrangement still remains in the state in which the chips 40 are arranged. This is for alignment. This alignment is based on coordinate information added to the instruction code, for example, with the Y coordinate as a reference. Furthermore, when generating the program, the instruction table 50 is referenced to confirm whether the order of the instruction codes is correct. For example, as a result of the above image processing, if the command code for the parameter "count" is lined up to the right of the command code for "move", it is correct. are lined up, the command table 50 does not permit it, so it is determined to be incorrect. If the alignment is incorrect, one example ignores the parameter. Also, even if the arrangement of the parameters for the instruction code is correct, if the instruction code of the parameter is shifted to the right beyond the allowable range, the instruction code is moved to the left and, for example, a "move" instruction is executed. Align right next to the code.

After the instruction codes are aligned and confirmed, the aligned instruction codes are coded in order to generate program data (S1404). The created program data is stored in the “program” area of the program management table 110 . The program management table 110 also registers a user identification ID, a curriculum number, and a program registration date and time.

A program based on the pattern arrangement of the chips 40 is created by the above processing operations. For example, as shown in FIGS. 7A to 7C, if the same subject is used but the arrangement of the chips is different, the instruction code arrangement is also different, resulting in a different program. However, the actions of the characters displayed on the screen of the display unit 205 by executing the program are the same.

[Run program]
In Example 1, the created program is executed by the CPU 202 of the mobile terminal 2 owned by the teacher. It is assumed that the execution of the program is displayed on the screen of the display unit 205 as a moving image accompanied by the action of the character. However, the program is not limited to this, and can be executed on various devices.

FIG. 16 shows devices (referred to as connected devices) connected to the server 1 in the program creation support system. For example, the server 1 is connected via the network 9 to devices including a projector for projecting moving images and images on a screen, a large-sized display device 151, an audio device 152, and an action device 153 such as a robot or minicar.

The connected device on which the program is executed is not limited to one, but can be multiple devices. For example, when "play sound" is used from among the input/output commands in the command list shown in FIG. As another example, in the case of using a robot that emits sound, if the input/output commands of the command list shown in FIG. The robot operates as the program is executed, and sounds are emitted from the speaker provided in the robot.

The connection device can be all devices having an IP address that can be connected to the Internet. For management of connected devices, a connected device management table for registering the IP addresses or URLs of connectable devices is prepared in the management information DB 105 of the server 1, and the connected device of the output destination is selected by referring to the connected device management table. By selecting it, it is possible to connect to the device on which the program is executed. The connection device may be selected in advance, or may be selected from the mobile terminal 2 owned by the teacher or another mobile terminal after the program is completed.

[Program creation support system for the visually impaired]
Although the above embodiment 1 assumes that a visually impaired person creates a program, the present invention is also useful when a visually impaired person creates a program. For example, the sheet 30 and each chip 40 are embossed in Braille for the visually impaired. That is, the matrix lines of the sheet 30, the guideline 304, the title 301 and the "start" mark 303 are printed in braille in addition to the above printing. Further, at a predetermined position of each chip 40, Braille representing the "name" of the chip is applied in addition to the above-described pattern. Further, in the command table 50, in addition to the configuration shown in FIG. 5, musical note data is registered corresponding to the "name" of each chip.

The user relies on the sheet 30 and Braille on each chip 40 to program the chips 40 side by side on the sheet 30 . The operation from acquiring the chip array sheet image with the camera to generating the program data is the same as in the first embodiment. In executing the program, the program can be executed by enabling the display function and the audio output function of the mobile terminal 2, or by connecting to the audio equipment 152 shown in FIG. In the execution of the program, the state of the arrangement of multiple chips is output as changes in the sounds of musical notes, so visually impaired people can grasp the performance of the program based on the changes in the sounds.

[Program creation support system according to another example]
In the program creation support system according to the first embodiment, the server 1 has a program creation processing unit 103, an image processing unit 104, a management information DB 105 storing related tables and information, and a program DB 106. FIG. However, in the programming support system according to the second embodiment, the portable terminal 2 can have all or part of these processing functions and DB.

For example, when all the processing functions and related tables and information are provided in the mobile terminal 2, first access a site that provides a programming creation service (for example, a site operated by the server 1) and use the related application (program creation functions and related management information) and stored in the mobile terminal 2. After that, a program can be created in the same manner as in the above embodiment. In this case, the sheet 30 and the chip 40 can be created by printing with a printer designated by a print instruction from the mobile terminal 2 . The processing shown in FIGS. 13 and 14 is executed by a program in the CPU 202 of the mobile terminal 2. FIG.

Further, as an example in which the mobile terminal 2 has a part of the processing functions and the DB, for example, the server 1 holds the command table 50, and the mobile terminal 2 has a curriculum management table 80, an image processing function, and a program creation function. can have a function. In this case, pairs of commands and images used according to the curriculum are obtained from the command table 50 and stored in the curriculum management table 80 . The program creation function can refer to the curriculum management table 80 and perform a process of specifying an instruction corresponding to the image of the chip. Usually, the mobile terminals 2 of many users can be connected to the server 1 . Therefore, all pairs of commands and images are managed as a population in the command table 50, and commands and images selected from the command table 50 are selected according to the curriculum specified by the request from the mobile terminal 2 of the user. It makes sense to provide only the pair to the mobile terminal 2 in question.

[Other examples of programming behavior]
In Example 1, as shown in FIG. 12, the arrangement of chips 40 on a sheet 30 prepared by a student is photographed using the mobile terminal 2 possessed by the teacher. However, the learning of programs is not limited to school classes, and users can also learn by themselves. According to the third embodiment, the user owns the portable terminal and can perform basic operations such as login by himself/herself.

As shown in FIG. 17, the user himself/herself logs in using a mobile terminal (S1201), selects a curriculum (S1202), prepares sheets and chipsets, and performs programming (S1211'). In this case, the user management table 90 is used instead of the teacher management table 100. FIG. Other main processing operations are the same as in the first embodiment.

In addition to program creation learning, this program creation support system is also applicable to game devices for creating programs, program creation for control of control devices and toys by programs, and the like.

[Program creation support processing flow according to another example]
In the first embodiment, in the processing flow with reference to FIG. 13, the process from capturing and transmitting an image (S1303) to creating a program (S1305) and executing the program (S1307) is performed as a series of processes. According to the fourth embodiment, it is also possible to divide these processing steps into a plurality of steps and perform them at different timings. For example, in a school class, in the first lesson, students are taught to arrange chips 40 on a sheet 30, and the chip arrays created by a plurality of students are photographed by the camera 206, and a plurality of chip array sheet images are stored in memory. 203, and the second time, the chip arrangement sheet image stored in the memory 203 is transmitted to the server 1, the server performs program creation processing, and the created program is transmitted to the portable terminal 2 and stored in the memory. 203, until one student's program is tested, and in the third test, the programs created by the remaining students are tested, and then the program is corrected. It is possible to implement. Especially in school classes, it is important to proceed while observing the progress of all students, so it is meaningful to divide the program creation process into several steps as described above.

[Management table by another example]
In the first embodiment, the curriculum management table 80 registers reference numbers of all commands used in the curriculum (reference numbers of commands selected from the command table 50 of FIG. 5). In the fifth embodiment, the chip image, name, operation, parameter, and instruction code data of the instruction selected from the instruction table 50 may be registered. Furthermore, by downloading an application for supporting program creation from the server 1 to the mobile terminal 2, the CPU 202 of the mobile terminal 2 can create a program based on the chip array image.

As another example of the fifth embodiment, a plurality of instructions selected from an instruction table 50, a sheet 30, and a chip 40 are provided in response to a user's request to ultimately assist the user in creating a program. business can be considered. In this case, the curriculum management table 80 becomes a contract management table for each user, and the curriculum number becomes a contract identification code unique to the user. For example, the information management unit 102 of the server 1 includes a contract management table in which each data of the chip image, name, operation, parameter, and instruction code of the selected instruction is registered, and the user A management table, a program management table, and an application for supporting program creation are provided to the mobile terminal 2 of the user. In the mobile terminal 2, the CPU 202 can create a program based on the acquired chip arrangement image by the processes shown in FIGS. 13 and 14. FIG.

[Sheet with other examples]
The seat 30 is not limited to that shown in FIG. Considering the ease and convenience of arranging the chips 40, it is preferable that the program area 302 has a "start" mark, ruled lines, and guidelines 304 printed thereon, but according to an alternative, they are not necessarily required. If the starting position and direction of arrangement of the chips 40 are known, or if the user is somewhat skilled, the ruled lines and guide lines 304 may be omitted in the program area 302 . Also, the "start" mark 303 can be substituted with a "START" chip.

Also, according to another example, sheet 30 is not necessarily required. In short, by clarifying the start position, the arrangement direction, and the end position according to the rules for arranging the chips 40, arranging the chips 40 on a blank sheet of paper, a desk, or a table, and photographing the arrangement with a camera, it is possible to achieve the same effect as in the first embodiment. A similar action or effect can be obtained. An image acquired by the camera is a chip arrangement image without a sheet. For example, when arranging chips on a desk, there is no problem even if there is a foreign object such as an eraser or a pencil on the desk. This is because in the process of image recognition based on images taken by a camera, images of erasers and pencils are excluded, and only images unique to the chip can be recognized and extracted.

As another example that does not use a sheet, for example, the chips 40 are made of quadrilateral cardboard cards about 1 to 2 meters in length and width, and the cards are arranged in the schoolyard for programming. It is also possible to acquire an image of the chip array by shooting from a high place with a camera equipped with a wireless communication function mounted on the drone.

[Example of assignment sheet]
Example 7 is a specific example of the assignment (curriculum) sheet shown in FIG. 15(B). As shown in FIG. 18, the task sheet 70 mainly contains a task sentence 702 and a picture, and has a task number 701 and an identification section 705 for managing the task (or task sheet). The identification part 705 is formed of a two-dimensional barcode including identification information (task identification information) for identifying the task. Assignment identification information is associated with a curriculum number registered in the curriculum management table 80 .

In light of the first embodiment, the teacher (or user) distributes the assignment sheet 70 corresponding to the assignment to the students each time the assignment changes. Then, when the teacher (or the user) distributes (uses) the assignment sheet 70 (for example, at S1204), the barcode reader 207 of the portable terminal 2 is used to read the identification portion 705 of the assignment sheet 70 . Further, in the program creation process (FIG. 13), after the application is started (step S1302), instead of having the barcode reader 207 read the identification section 305 of the sheet 3 (in the first embodiment), the task sheet 70 can be identified. The part 705 is read by the barcode reader 207 . As a result, the assignment sheet 70 and the curriculum number registered in the curriculum management table 80 are associated with each other. Other operations are the same as those of the first embodiment, and can be understood by replacing the identifying section 305 in FIG. 13 with the identifying section 705 .

When this task sheet 70 is used, the sheet on which the chips 40 are arranged may not have the identification section 305 . FIG. 19 shows an example of this sheet 30'. In other words, according to the first embodiment, the sheet 30 is changed every time the curriculum changes. ' does not need to be changed. Therefore, the sheet 30' can be used as a dedicated sheet for arranging the chips 40 (for example, a desk mount).

As an application example of the seventh embodiment, a booklet containing a multi-page task sheet 70 is provided to the user for a charge or free of charge. In this case, the identification unit 705 preferably includes site information of the server 1 that provides the programming service, in addition to the assignment identification information. The user causes the bar code reader 207 of the mobile terminal 2 to read the identification part 705 of the target assignment sheet 70, accesses the site of the programming service based on the read information, and responds to the assignment sheet 70. The information registered in the curriculum management table 80 can be obtained, and the program creation service can be received.

[Tips according to other examples]
In the eighth embodiment, unevenness is provided on the outer peripheral portion of the chip 40 in order to facilitate the arrangement of the chips 40 . As shown in FIG. 20, a flat chip 40 has triangular concave portions 411 and 412 formed on its upper and left sides, and triangular convex portions 421 and 422 formed on its right and lower sides. The reason why the unevenness is provided on the chip 40 is to regulate the arrangement of the chips 401, in other words, to simplify the arrangement of the chips. As shown in FIG. 21, the user can easily arrange (or combine) a plurality of chips 40 in sequence by engaging the projections of one chip with the recesses of another chip. can be done. As a specific example, as shown in FIG. 22, a parameter can be added to the "move: with direction" command (see FIG. 5), so the chip 401 of the "move: with direction" command has a protrusion 422 on the right side. The parameter chip 402 has a recess 412 only on the left side. Although not shown in FIG. 22, as another example of unevenness of the chip with reference to FIG. None. Also, since all parameters are added to operation commands, control commands, etc., the parameter chip has a concave portion 412 only on the left side thereof, and does not have concave portions or convex portions on the other sides. Also, as shown in FIG. 22, the chip 401 for the "end" command has a concave portion 411 only on the upper side.

According to the eighth embodiment, by appropriately forming irregularities on the sides of the rectangular chip 40 and arranging a plurality of such chips, the chip arrangement of commands and parameters can be managed. As a result, the arrangement of the chips can be managed uniformly, so that the sheet 30 for arranging the chips can be omitted.

In addition, since the arrangement of chips can be managed, the time and load for executing the image correction function (S1401 (FIG. 14)) in image processing and the program creation processing code data alignment function (S1403) can be reduced. As a whole, the time for program creation can be speeded up. For example, when the chips 40 without unevenness are arranged on the sheet 30 as in the first embodiment, even if there is a guideline 304, the left and right sides of the plurality of chips are bumpy, biased, or tilted. They are often arranged. In such a case, normal correction can be performed by the image correction function (S1401), but processing time or load is required. On the other hand, according to the eighth embodiment, since the arrangement of the chips is managed by the unevenness provided on the chips, it is possible to prevent unevenness and inclination of the arrangement of the chips by the user. Therefore, processing such as the image correction function (S1401) by the image processing program is almost unnecessary, and the speed of program creation processing can be increased.

According to the eighth embodiment, triangular unevenness is formed on the sides of the rectangular chip, but the shape of the unevenness is not limited to the triangular shape, and various shapes such as a semicircular shape and a square shape can be applied. Also, if only certain parameters are suitable for a certain instruction, both chips can be applied with specific shapes of bumps. For example, a "jump" command is only given parameters of "orientation and height", so both chips have a specific relationship. Therefore, while the triangular unevenness is applied to the chips of the majority of instructions, special unevenness (for example, half Circular unevenness) can be applied. In this way, it is possible to prevent chips with "orientation, height" parameters from being selected for chips with other motion commands.

[Tips according to other examples]
As another example of the chip 40, the pattern, shape, and size of the chip 40 are not limited to those shown in FIGS. As for the pattern, it is not limited to the abstract picture or characters shown in FIG. 4, but may be a concrete picture such as a rabbit running, spinning, or picking up an object. This would make it easier for infants to understand the relationship between the picture and the meaning of the command. Other examples include playing cards and karuta. Moreover, the shape of the chip is not limited to a quadrilateral, and may be various shapes. For example, it may be a shape cut out of a picture of a rabbit running, sleeping, or flying.

In addition, although the chip 40 is referred to as the chip 40 in the first embodiment, the chip 40 is not limited to this. For example, cards, parts, pieces, strips, pieces of paper, and elements may be collectively called parts or articles.
The chips are preferably planar cards for the convenience of arranging them and then putting them away for storage, but they are not necessarily limited to planar members. In short, it can be realized as long as it is an item that can be arranged, and the characteristic of the pattern or shape of the item (referred to as visual recognition characteristic) is uniquely associated with the instruction. It should be noted that an article with visible characteristics may be referred to as a part with a unique indicia.

Further, in the above embodiment, the pattern of the chip 40 is associated with the instruction by image recognition, but the present invention is not limited to this. For example, each chip 40 may be given a unique number or bar code in addition to a pattern, and by recognizing the number or bar code, it is possible to associate it with an instruction. In that case, the relationship between numbers or barcodes and commands is registered in the command table shown in FIG.

As another example of the chip 40, the chip can be created and used by the user himself. For example, the user can increase the number of chips by hand-making or copying chips 40 having the same shape and pattern as those in FIG. In this case, if the pattern of the chip prepared by the user is the same as the predetermined one or within the allowable range, the pattern of the chip 40 prepared by the user is recognized as the same image in the image processing of the server 1, so there is no problem. do not have.

[Other examples]
In the above embodiment, the identification portion 305 of the sheet 30 is read by the barcode reader 207 of the mobile terminal 2 . Alternatively, the camera 206 can be used to read the identification section 305 and the identification information held by the identification section 305 can be recognized by the image recognition function.

In the above embodiment, the camera 206 of the mobile terminal 2 is used to photograph the array of chips arranged on the sheet. According to an alternative example, an image of the chip arrangement taken by a camera having a communication function or a normal digital camera is stored in a USB memory or an SD memory, and a program is created in this memory. It is also possible to obtain an image of the chip array by loading it into a server or terminal for the purpose.
The present invention can be practiced with various modifications and applications other than the above-described embodiments and alternatives.

1: Server 2: Portable Terminal 9: Network 102: Information Management Unit 103: Program Creation Processing Unit 104: Image Processing Unit 105: Management Information DB 106: Program DB
202: CPU 203: Memory 204: Input section 205: Display section 206: Camera 207: Barcode reader 208: Network connection section 30: Sheet 301: Title 302: Program area 303: Start mark 304 Guideline 305: Identification section 40: Chip 50: Instruction table 70: Assignment sheet 80: Curriculum management table 90: User management table 100: Teacher management table 110: Program management table

Claims (5)

A program creation support system that supports creation of a program using a terminal having an input unit, a display unit, a processor that executes a program, and a storage unit,
A plurality of articles are used, each of which has a unique code and includes a start article that indicates a start mark that serves as a starting position, a plurality of action articles that define the action, and an end article that indicates an end mark that serves as an end position,
The program has a curriculum presented to a user for programming, in which at least task texts and pictures are described, and curriculum identification information for identifying the curriculum, the start item, the plurality of action items, and the end item. Curriculum sheets are used that do not have areas in which items are placed,
a curriculum management table for managing the correspondence between the code and instructions of each of a plurality of items used for programming based on the curriculum, according to the curriculum identification information for identifying the curriculum for programming;
Acquisition means for acquiring the code of each of a plurality of items arranged in a place other than the curriculum sheet by the user according to the assignment text and the picture diagram of the curriculum of the curriculum sheet;
a code recognition processing unit that recognizes the code of each of the plurality of articles acquired by the acquisition means;
a program creation processing unit that refers to the curriculum management table, identifies instructions corresponding to the code recognized by the code recognition processing unit, and generates a program composed of the identified instructions; has
selecting the curriculum management table corresponding to the curriculum designated by the user's operation from the terminal;
The acquiring means acquires the code of each item in an array formed from the starting item, the plurality of action items arranged after the starting item, and the end item,
The code recognition processing unit recognizes the code of the plurality of action items arranged between the start item and the end item,
The program creation processing unit refers to the selected curriculum management table and creates a program composed of the instructions corresponding to the codes recognized by the code recognition processing unit. support system. the terminal has a barcode reader, the barcode reader reads the curriculum identification information on the curriculum sheet;
2. The program creation support system according to claim 1, wherein said curriculum management table is selected according to said read curriculum identification information. 2. The programming support system of claim 1, wherein each of said plurality of items has a unique visual characteristic and said unique code. A program creation support method for assisting creation of a program using a terminal having an input unit, a display unit, a processor for executing a program, and a storage unit,
A plurality of articles are used, each of which has a unique code and includes a start article that indicates a start mark that serves as a starting position, a plurality of action articles that define the action, and an end article that indicates an end mark that serves as an end position,
The program has a curriculum presented to a user for programming, in which at least task texts and pictures are described, and curriculum identification information for identifying the curriculum, the start item, the plurality of action items, and the end item. Curriculum sheets are used that do not have areas in which items are placed,
a step of managing, in a curriculum management table, the correspondence between the codes and instructions possessed by each of a plurality of articles used for programming based on the curriculum, according to the curriculum identification information for identifying the curriculum for programming;
an obtaining step of obtaining the code of each of the plurality of items arranged in a place other than the curriculum sheet by the user according to the assignment text and the diagram of the curriculum of the curriculum sheet;
a code recognition processing step for recognizing the code of each of the plurality of articles acquired by the acquiring step;
a program creation processing step of referring to the curriculum management table to identify instructions corresponding to the codes recognized in the code recognition processing step, and generating a program composed of the identified instructions; has
selecting the curriculum management table corresponding to the curriculum designated by the user's operation from the terminal;
the acquiring step acquires the code of each item in an array formed from the starting item, the plurality of action items arranged after the starting item, and the end item;
The code recognition processing step recognizes the codes of the plurality of action items arranged between the start item and the end item,
The program creation processing step refers to the selected curriculum management table and creates a program composed of the instructions corresponding to the codes recognized in the code recognition processing step. how to help. a barcode reader of the terminal reads the curriculum identification information on the curriculum sheet;
5. The program creation support method according to claim 4, wherein said curriculum management table is selected according to said read curriculum identification information.

Images