JP3795322B2 - Operation history device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
  The present invention relates to an operation history device that stores a history of operations performed by an operator.
[0002]
[Prior art]
  For example, in a factory, the system is operated by an operator operating an operation unit such as a control panel. When an abnormality occurs in the operation of the system, it is important to identify the cause of the occurrence of the abnormality in order to restore the normal operation state. In particular, when there is a high possibility that an abnormality occurs due to an operator's erroneous operation, it is necessary to clarify what kind of operation caused the abnormality. For this reason, it is necessary to record the operator and the operation of the operator by some method.
[0003]
  Conventionally, as the recording, for example, an operation panel such as a programmable display is always photographed with an operator with a video camera, and the video is recorded on a VTR and the time stamp is recorded together with the video. A method of leaving a history of operations performed by the operator on the side was employed. If an abnormality occurs, the video at the time when the abnormality occurred, which is specified by the time stamp, is reproduced on the VTR, and the operator who has operated the operation panel is identified from the video, and the operation The operation at that time is specified from the operation history of the panel.
[0004]
[Problems to be solved by the invention]
  However, in the recording method as described above, since the operation history is stored on the operation panel side independent of the recording device, it is necessary to separately specify the operator on the recording device and the operation on the operation panel. Both of them are complicated to specify. For this reason, it takes time to verify the cause of the abnormality, and in particular, if the cause of the abnormality cannot be identified and the normal state cannot be restored, the time required for the restoration is prolonged and the damage is increased. .
[0005]
  Further, in the above recording method, an image must be recorded for a long time using a recording device such as a VTR because of an erroneous operation that does not know when it occurs. That is, most of the recorded video is not used, and unnecessary recording is performed. For this reason, it is necessary for the recording medium to have an enormous recording capacity, which disadvantageously increases the cost for recording.
[0006]
  The present invention has been made in view of the above circumstances, and has as its main purpose to simplify the specific work of both by associating and storing the operator and the operation content, and further, It is intended to efficiently save the history of users and operations.
[0007]
[Means for Solving the Problems]
  In order to solve the above problems, an operation history device of the present invention includes a time generation unit that generates a time, an image reading unit that reads an image of an operator who has operated the operation device, and an operator using the operation device. When the operation is performed by the operation detecting means for detecting information about the operation performed by the image storage means, the image storing means for storing the image read by the image reading means, the operation information detected by the operation detecting means, and the operator An operation information storage means for storing the time generated by the time generation means together with related information related to the image stored in the image storage means;A change amount determination for determining whether or not a change amount of the read image with respect to the stored image is larger than a predetermined value by comparing the image previously stored in the image storage unit and the image read by the image reading unit And a storage permission unit that permits storage of the read image in the image storage unit only when it is determined that the amount of change is greater than a predetermined value and the operation information is detected.It is characterized by having.
[0008]
  In the above configuration, the image of the operator (particularly the operator's face) read by the image reading unit is stored in the image storage unit, while the operation information detected by the operation detection unit and the time generated by the time generation unit Is stored in the operation information storage means together with related information. Thereby, the image memorize | stored in the image memory | storage means is linked | related by operation information and time by related information. As this related information, for example, when the image storage means is an image memory, an address used for writing and reading image data to and from the image memory is used. With the association as described above, the details of the operation performed and the time when the operation was performed can be easily specified together with the image of the operator.Further, a change amount for determining whether or not a change amount of the read image with respect to the stored image is larger than a predetermined value by comparing the image previously stored in the image storage unit with the image read by the image reading unit. A determination unit, and a storage permission unit that permits storage of the read image in the image storage unit only when it is determined that the amount of change is greater than a predetermined value and the operation information is detected. The read image is stored in the image storage means by the storage permission means only when the change amount determination means determines that the change amount of the read image with respect to the stored image is larger than the predetermined value and the operation information is detected. Thus, when the operator does not change and the change in the read image with respect to the stored image is small, the read image is not stored, so that an extra image with a small change is not stored.
[0009]
  Preferably, the operation history device further includes a combining unit that combines the image stored in the image storage unit with the operation information and time stored in the operation information storage unit. As described above, the image, the operation information, and the time are combined by the combining means, and the image obtained as a result of the combination includes the image of the operator, the operation information by the operator, and the time when the operation was performed. And are included. This makes it possible to specify the operator, operation content, and operation time simply by looking at the synthesized image.The
[0010]
  UpIn the operation history device described above, it is preferable that the change amount determination means compares the two images with the sum of the data of all the pixels of the image. Thus, by comparing with the total sum of all pixel data, the comparison object is simplified, and the comparison process can be easily performed.
[0011]
DETAILED DESCRIPTION OF THE INVENTION
  One embodiment of the present invention will be described with reference to FIGS. 1 to 5 as follows.
[0012]
  As shown in FIG. 1, the operation history apparatus according to the present embodiment includes a video camera 1, an image processing unit 2, a key operation detection unit 3, an RTC time stamp generator 4, an image memory 5, a history memory 6, and an image comparison. A unit 7, an AND gate 8, an address generation unit 9, a FIFO (First-IN First-OUT) memory (FIFO in the figure) 10, and an image composition unit 11 are provided.
[0013]
  The video camera 1 as an image reading unit will be described later.Operation unit 21Is provided for photographing the face of an operator who operates the camera, reads images that are constantly photographed, and outputs them as video signals (composite video signals) such as analog NTSC and PAL formats. The video camera 1 may be a camera that outputs the read image as digital data.
[0014]
  The image processing unit 2 performs various processes on the video signal from the video camera 1. The processing includes, for example, video decoding processing that extracts an image signal from a video signal, converts the extracted image signal to digital, and separates a horizontal synchronizing signal and a vertical synchronizing signal from the video signal. And processing such as compression, reduction, and feature extraction for image signals. In the compression and reduction processing, when it is necessary to compress or reduce an image, the storage area of the image data in the image memory 5 is reduced by performing the processing before the writing of the image data into the image memory 5. It can be effective. The feature extraction process is, for example, a process of extracting an image feature (such as the contour of the operator's face) as feature data, and is used when only the image feature is stored.
[0015]
  Regarding the processing performed by the image processing unit 2, a necessary function is determined according to the design specifications of how the image is stored.
[0016]
  The key operation detection unit 3 as an operation detection unit detects that the operator has performed a key operation (operation of keys, switches, buttons, etc.) by the operation at the operation unit 21, and as a result of the detection, the key operation is performed. The key entry signal KEY that becomes a high level at the timing of and the key code data KEYCODE (multiple bits) representing the operated key are output. The key entry signal KEY is a signal that is active (for example, at a high level) for at least a predetermined time from the start of key operation to the completion of writing of image data for one screen as described later.
[0017]
  The operation unit 21 as an operation device is an operation panel or the like provided in various devices, and a keyboard, operation switches, operation buttons, a touch panel, a mechanical switch, and the like are provided as operation means. The device provided with the operation unit 21 is preferably a control operation device such as a programmable display, but may be a device other than the control device such as an interphone.
[0018]
  The RTC time stamp generator 4 as time generating means generates a time stamp as digital data of date and time (year / month / day and time) by a real time clock.
[0019]
  An image memory 5 as an image storage means is a semiconductor memory (for example, DRAM) that stores digital image data that has undergone processing in the image processing unit 2, and image data for addresses generated by the address generation unit 9 at the data terminal DATA. Write. The image memory 5 reads the written image data in accordance with the display timing on the display device 22.
[0020]
  The history memory 6 as the operation information storage means includes data relating to the key operation history, that is, the key code data KEYCODE from the key operation detector 3, the time stamp data TIME from the RTC time stamp generator 4, and the address generator 9. This is a semiconductor memory (for example, DRAM) that stores the above-described addresses (image addresses) generated in the above. The history memory 6 writes these data input from the data terminal DATA and reads the written data in accordance with the display timing on the display device 22. The history memory 6 gives the read image address to the image memory 5. The image memory 5 reads the image data specified by the image address given the image address.
[0021]
  The image memory 5 and the history memory 6 may be configured by a data storage device other than the semiconductor memory, for example, a hard disk device. However, when this history control device is installed on a production site such as a factory line, it operates stably even in harsh environments where vibrations, dust, etc. are generated. A semiconductor memory is more suitable than that.
[0022]
  As shown in FIG. 2, the history memory 6 stores a key code (key code data KEYCODE), a time stamp (time stamp data TIME), and an image address as one set. The image address is an all address used when data (pixel data) of all pixels constituting the image is written to or read from the image memory 5 one by one. This image address serves as related information for associating key code data KEYCODE and time stamp data TIME with image data. The storage capacities allocated to the key code, time stamp and image address are 2 bytes, 6 bytes and 4 bytes, respectively.
[0023]
  When new image data is not stored in the image memory 5, that is, when almost the same image data is input, the same image address is included in different data sets. For example, the data set “1” when the image is stored and “2” when the image is not stored correspond to the image “1” stored in the image memory 5 and include the same image address. It is out. The data set “3” corresponds to the newly stored image “3”.
[0024]
  The image comparison unit 7 compares the image (read image) data input from the image processing unit 2 with the image data previously written in the image memory 5, and changes the input image data with respect to the write image data. A low level signal is output when the amount is less than or equal to a predetermined value, while a high level signal is output when the amount of change is greater than a predetermined value. Therefore, the image comparison unit 7 determines whether or not the amount of change is greater than a predetermined value.
It functions as a change amount determination means.
[0025]
  For the comparison performed by the image comparison unit 7, for example, a method of comparing the sum of the pixel data constituting the image data for one screen with the input image data and the write image data is suitable. In this method, first, the sum of all the pixel data Aij represented by i = 0 to n and j = 0 to m of the pixel data Aij in the i-th column and the j-th row in the input image shown in FIG. ΣAij and the sum ΣBij of all pixel data Bij represented by i = 0 to n and j = 0 to m of the pixel data Bij in the i-th column and the j-th row in the written image shown in FIG. And the difference between the two is compared with a predetermined threshold. In this embodiment, this comparison method is employed, and details thereof will be described later.
[0026]
  Other comparisons include a method of comparing both image data one pixel at a time and comparing them with a threshold value with high accuracy. However, these require a lot of arithmetic processing, so that the processing time is prolonged and the circuit becomes complicated when it is configured by hardware. On the other hand, if high accuracy is not required, the above method shown in FIGS. 3A and 3B can most easily perform comparison and size determination.
[0027]
  The AND gate 8 serving as a storage permitting unit is a logic circuit that outputs a logical product of the output signal from the image comparison unit 7 and the key entry signal KEY from the key operation detection unit 3. A signal output from the AND gate 8 is used as a write control signal WRITE that permits writing to the image memory 5.
[0028]
  The address generator 9 is based on the high level output signal from the AND gate 8.
Generate an address. The address generator 9 will be described in detail later.
[0029]
  The FIFO memory 10 is a memory that stores image data output from the image processing unit 2 for one screen, and sequentially writes the image data in units of one pixel, and shifts the data to the output terminal in the order in which the data is written. read out. The FIFO memory 10 outputs a high level read permission signal from a read permission terminal DOR (Data Out Ready) when the pixel data is shifted to the final stage register and pixel data for one screen is written.
[0030]
  The image synthesizing unit 11 as a synthesizing unit synthesizes the image data read from the image memory 5 and each data read from the history memory 6 and outputs the synthesized data to the display device 22. As a composition method, the time stamp and the name of the operated key are superimposed on the image for one screen displayed on the display device 22, or the time stamp and the key name are combined. A method is conceivable in which the images are combined so as to be displayed in a dedicated display area provided separately from the display area. However, the synthesis is not limited to these methods, and a desired method is adopted.
[0031]
  The display device 22 is a display device that displays the image synthesized by the image synthesis unit 11. The display device 22 may be provided as a dedicated display device, but may be a display unit of the aforementioned programmable display.
[0032]
  Next, the image comparison unit 7 and the address generation unit 9 of the operation history device will be described in detail.
[0033]
  As shown in FIG. 4, the image comparison unit 7 includes an AND gate 701, an inverter 702, a delay circuit 703, an adder 704, a latch 705, image latches 706 and 707, a subtracter 708, a complementer 709, a multiplexer 710, and a latch 711. And a comparator 712.
[0034]
  The AND gate 701 outputs a logical product of the valid signal VALID and the pixel clock PCLK. As shown in FIGS. 5A and 5B, the pixel clock PCLK is a clock synchronized with the input timing of pixel data for one pixel in the input image, and the valid signal VALID is a video signal in one horizontal period. It is a signal that becomes a high level during the period of the image signal (image effective period) that represents only the image excluding the synchronization signal and the like.
[0035]
  The inverter 702 inverts the vertical synchronization signal VSYNC (see FIG. 5A). The vertical synchronization signal VSYNC is extracted from the video signal from the video camera 1 by the video decoding process of the image processing unit 2 and supplied to the image comparison unit 7. The delay circuit 703 is a circuit that delays the output signal from the inverter 702 for a predetermined time.
[0036]
  The AND gate 701 outputs a logical product of the key entry signal KEY and the output signal from the comparator 712. As will be described later, the output signal of the comparator 712 is a difference between the total data held in the image latch 706 (corresponding to input image data) and the total data held in the image latch 707 (corresponding to write image data). Is a signal that becomes high level when the value exceeds the predetermined value.
[0037]
  The adder 704 adds the output data from the latch 705 to the input pixel data PIX (see FIG. 5B). The latch 705 latches and outputs the addition data from the adder 704 at the timing of the output pulse of the AND gate 701. The latch 705 is reset every vertical period by the output pulse of the delay circuit 703. As a result, the sum of pixel data for one screen is output from the latch 704. In addition, since the reset timing is delayed from the normal vertical synchronization signal VSYNC by the delay circuit 703, the total data is surely latched in the image latch 706 before being erased by the reset of the latch 705. .
[0038]
  The image latch 706 latches the total data input as the evaluation value ESTA of the input image data during the period of two pulses following the vertical synchronization signal VSYNC input to the clock terminal. On the other hand, the image latch 707 latches the sum data corresponding to the image data written to the image memory 5 among the sum data output from the image latch 706 as the evaluation value ESTB of the image data. In the image latch 707, the write control signal WRITE from the AND gate 8 is input to the clock terminal, one image data is written in the image memory 5, and the next image data is written in the image memory 5. Until the total data is latched. Therefore, the total data output from the image latch 706 during that time is not latched by the image latch 707.
[0039]
  The above-described image latches 706 and 707 have the same low active reset signal RST.^* Reset by. This reset signal RST^* The system reset signal of the device incorporating the present operation history device or the operation history device is used, and is output when starting these devices or operating the reset switch.
[0040]
  The subtracter 708 subtracts the sum data of the image latch 706 from the sum data of the image latch 707 and outputs the result. The subtractor 708 outputs a high level borrow signal from the borrow terminal BOR indicating that the subtraction value has been borrowed when the subtraction result (subtraction value) is a negative value.
[0041]
  The complementer 709 is a circuit that calculates and outputs the complement of subtraction data (subtraction data) from the subtractor 708, and includes an inverter 709a and an adder 709b. The complementer 709 obtains a 1's complement for the subtraction data by inverting the subtraction data by the inverter 709a, and further, the adder 709b obtains the inverted subtraction data and 1 (“00001” in the case of 5 bits). The 2's complement for the subtraction data is obtained by addition. The complement output from the complementer 709 represents the negative value of the subtraction data.
[0042]
  The multiplexer 710 selects either the complement data of the complement 709 input to the “1” input terminal or the subtraction data from the subtractor 708 input to the “0” input terminal, and outputs it as difference data. To do. Specifically, the multiplexer 710 outputs complement data when the borrow signal is at a high level, and outputs subtraction data when the borrow signal is at a low level.
[0043]
  The latch 711 latches the threshold data supplied from the CPU 23 at a predetermined clock CLK timing and outputs the latched data to the comparator 712. The clock CLK is a clock signal that is output by one clock only when the CPU 23 accesses the latch 711. Therefore, the clock CLK is obtained as a logical product of a signal obtained by decoding the address of the CPU 23 by an address decoder (not shown) and an access signal output when the CPU 23 accesses the latch 711. As a result, the threshold data is latched in the latch 711 until there is a new clock input. Further, the CPU 23 updates the threshold data when the threshold data is updated.
An access signal is output so that new threshold data is output to the H.711.
[0044]
  The comparator 712 compares the difference data from the multiplexer 710 input to the input terminal X with the threshold data from the latch 711 input to the input terminal Y, and the difference data is equal to or less than the threshold data. A low level signal is output to the output signal, and a high level signal is output when the difference data is greater than the threshold data.
[0045]
  In the image comparison unit 7 shown in FIG. 4, as described above, the image latches 706 and 707 perform the process of latching the evaluation values ESTA and ESTB and comparing the two values, respectively, and therefore, as shown in FIG. 1. Thus, the previous written image data from the image memory 5 is not compared with the input image data.
[0046]
  The address generation unit 9 includes an adder 91, a latch 92, and a counter 93.
[0047]
  The adder 91 adds the output data from the latch 92 to the input frame size data FSIZE. The latch 92 latches and outputs the addition data from the adder 91 at the timing of the write control WRITE from the AND gate 8. The frame size data FSIZE is a capacity occupied by the image data for one screen in the image memory 5, and this value is used as a head address when the image data for one screen is written in the image memory 5.
[0048]
  The latch 92 is similar to the image latches 706 and 707 described above, and the reset signal RST.^* Reset by.
[0049]
  The counter 93 is included in the controller 14 that controls writing and reading of the image memory 5. The counter 93 is a counter that counts the write clock WCLK input to the clock input terminal. When a high level read permission signal from the FIFO memory 10 is input to the load terminal LD, the counter 93 is input to the data input terminal D. The output data from the latch 92 is set (preset), and counting is started from the next value of the value of the output data. The count data output from the output terminal Q of the counter 93 is given to the image memory 5 as an address. The period during which the write clock WCLK is supplied to the counter 93 is limited so that writing is not performed during the reading of the image data from the image memory 5.
[0050]
  Here, the operation of the operation history device configured as described above will be described.
[0051]
  The video camera 1 captures the face of the operator who is operating the switch of the operation unit 21 and the like. The image of the operator's face captured from the video camera 1 is subjected to predetermined processing by the image processing unit 2 and input to the image comparison unit 7 as digital image data, and is temporarily stored in the FIFO memory 10. Is input to the image memory 5.
[0052]
  First, when an image is input for the first time, the image comparison unit 7 inputs pixel data PIX constituting image data for one screen to the adder 704 one by one, where the addition data and the addition data via the latch 705 are added. Is done. As a result of this addition being performed for an image for one vertical period, that is, one screen, the sum of the pixel data PIX is obtained, and the sum data is latched in the image latch 706 as the evaluation value ESTA of the input image data. At this time, the sum data is not latched in the image latch 707. Therefore, in the subtracter 708, when the evaluation value ESTA of the image latch 706 is subtracted from the evaluation value ESTB of the image latch 707 that is 0, the evaluation value ESTA is output as negative difference data.
[0053]
  At this time, since the borrowing terminal BOR of the subtracter 708 is at a high level, the difference data of the complement obtained by the complementer 709 is output as negative difference data via the multiplexer 710. This difference data is compared with the threshold data from the latch 711 by the comparator 712. Further, since the difference data is larger than the threshold data, a high level signal is output from the comparator 712. Since the high level signal is input to the AND gate 8, the output terminal of the AND gate 8 is also at the high level. As a result, the frame size data FSIZE output from the adder 91 is latched by the latch 92.
[0054]
  On the other hand, the pixel data PIX is sequentially written in the FIFO memory 10 while the above processing is performed. When pixel data PIX for one screen is written in the FIFO memory 10, a high-level read control signal is output from the read control terminal DOR. In response to this read control signal, the CPU 23 instructs the FIFO memory 10 to read. Upon receiving this instruction, the FIFO memory 10 reads the pixel data PIX stored in the last register and shifts the pixel data PIX written thereafter to the previous register one by one.
[0055]
  The counter 93 is loaded with the frame size data FSIZE from the latch 92 in response to the input of the read control signal, and starts counting from the next value of the frame size data FSIZE to generate an address. The loaded frame size data FSIZE is output from the counter 93 as a write start address. At this time, the image memory 5 is in a writable state by the output signal of the AND gate 8, that is, the write control signal WRITE. As a result, the pixel data PIX sequentially output from the FIFO memory 10 is written in the area specified by the address in the image memory 5.
[0056]
  On the other hand, in the history memory 6, the above address is written as an image address together with the key code data KEYCODE and the time stamp data TIME. Specifically, as shown in FIG. 2, when the image “1” is written in the image memory 5, the data set “1” is also written in the history memory 6.
[0057]
  Next, when a new image is input, the image comparison unit 7 adds the pixel data constituting the image data for one screen in the adder 704 in the same manner as described above. That is, the evaluation value ESTA of the input image data is obtained and latched in the image latch 706. In addition, after the above addition is performed for the image for one screen, the latch 705 is reset to prepare for the addition for the next image. On the other hand, the image latch 707 latches the sum data for the write image data written in the image memory 5 as the evaluation value ESTB of the write image data.
[0058]
  In the subtracter 708, the evaluation value ESTA is subtracted from the evaluation value ESTB to obtain a difference between the two. If the difference data is a positive value, it is input to the multiplexer 710 as it is. At this time, since the borrowing terminal BOR of the subtracter 708 is at a low level, positive difference data is output from the multiplexer 710. On the other hand, if the difference data has a negative value, the complement of the difference data is obtained by the complementer 709 and the result is input to the multiplexer 710 as negative difference data. At this time, since the borrowing terminal BOR of the subtracter 708 is at the high level, negative difference data represented by a complement is output from the multiplexer 710.
[0059]
  The difference data from the multiplexer 710 is compared with the threshold data from the latch 711 by the comparator 712. As a result of the comparison, when the difference data is equal to or less than the threshold data, that is, when the change of the input image with respect to the written image is small, a low level signal is output and the output terminal of the AND gate 8 is also low level. In this case, the clock terminal of the latch 92 in the address generation unit 9 remains at a low level, and the latch state of the addition data from the adder 91 is maintained, so that the head address from the latch 92 becomes the adder 91. Is not updated to the value of the addition data from.
[0060]
  On the other hand, when the above processing is performed, the pixel data PIX is sequentially written in the FIFO memory 10, and when the pixel data PIX for one screen is written, a high-level read control signal is output from the read control terminal DOR. The Then, the counter 93 is loaded with the non-updated head address from the latch 92 and generates an address by counting. The address generated at this time is the same as the address generated at the previous writing. The image memory 5 is incapable of writing because the write control signal WRITE from the AND gate 8 is at a low level in spite of the input of the address as described above. Thereby, the pixel data PIX sequentially output from the FIFO memory 10 is not written to the image memory 5.
[0061]
  In this case, as shown in FIG. 2, in the history memory 6, the above-mentioned non-updated address is written as an image address together with the key code data KEYCODE and the time stamp data TIME. That is, when the change of the input image data with respect to the write image data is small, the input image data is not written in the image memory 5, but the key code data KEYCODE, time stamp data TIME, and image address corresponding to the image are stored in the history memory. 6 is written. Specifically, as shown in FIG. 2, when the image “1” is written in the image memory 5, the data set “2” is written in the same manner as the data set “1” written in the history memory 6 at the same time. Is written in the history memory 6 corresponding to the image “1”.
[0062]
  On the other hand, as a result of the comparison by the comparator 712, when the difference data is larger than the threshold data, that is, when the change of the input image with respect to the written image is large, a high level signal is output and the output terminal of the AND gate 8 is also high. Become a level. In this case, since the new addition data from the adder 91 is latched when the clock terminal of the latch 92 in the address generator 9 changes to a high level, the head address from the latch 92 is updated. For this reason, when the counter 93 counts from a new head address, an address different from that at the time of writing the previous image data is output from the counter 93. Accordingly, the pixel data PIX read from the FIFO memory 10 is written in the area of the address in the image memory 5.
[0063]
  At this time, the above address is written in the history memory 6 as an image address together with the key code data KEYCODE and the time stamp data TIME. For example, as shown in FIG. 2, when the image “3” is written in the image memory 5, the data set “3” is also written in the history memory 6.
[0064]
  When reproducing the stored image, the image address read from the history memory 6 together with the key code data KEYCODE and the time stamp data TIME is input to the address terminal ADD of the image memory 5 although not shown in FIG. As a result, the image data is read from the image memory 5. The image data is combined with the key code data KEYCODE and the time stamp data TIME by the image combining unit 11 and given to the display device 22. As a result, the display device 22 displays an image including key operation information (operation information detected by the operation detection unit) represented by the key code data KEYCODE and date / time information represented by the time stamp data TIME. .
[0065]
  As described above, the operation history device uses the operator's image captured by the video camera 1, information on the operation performed by the operator using the operation unit 21, and the date and time (time) when the operation was performed. The information is stored together with the information, and the information is reproduced (displayed) together with the image. Thus, by reproducing the stored image, it is possible to reproduce which operator performed what operation and when. Therefore, in production sites, etc., even if a system malfunction occurs due to an operator's mistaken operation, it is possible to specify the operation content, operation date and time, and the operator who performed the operation by playing back the stored images. Is done. As a result, it is possible to easily confirm the cause of the occurrence of the abnormality, take measures for recovery from the abnormality at an early stage, and confirm the circumstances of such an operation to the operator at an early stage. Therefore, it becomes easy to verify a malfunction caused by an erroneous operation and an erroneous operation.
[0066]
  For example, when one control instruction is given by a combination of a plurality of switch operations, the operated switch is displayed together with an image of the operator's face. Thereby, an error in the switch operation can be easily confirmed.
[0067]
  Further, by incorporating this operation history device into an interphone, a visitor as an operator presses a switch on the intercom, so that an image of the visitor is stored together with switch operation information and operation date and time. Therefore, by reproducing the image, it is possible to confirm when and who has visited. In this configuration, since one switch is operated, there is no need to confirm, and it is not necessary to display the image together with the visitor image.
[0068]
  The operation history device is configured to store only an image that changes larger than a predetermined change amount with respect to an already stored image when the image is stored in the image memory 5. Thereby, the capacity required for storage in the image memory 5 can be reduced. In particular, at the production site and the like, the same operator may perform a number of operations in a short time, so the operator's image taken by the video camera 1 hardly changes. Therefore, it is possible to avoid storing the same operator image in duplicate.
[0069]
  In this operation history apparatus, the change amount of the input image data with respect to the write image data by the image comparison unit 7 is detected based on the comparison of the sum of all pixel data PIX constituting both image data. As a result, the processing for detecting the amount of change is performed by a simple arithmetic process of adding all the pixel data PIX and subtracting the total data obtained thereby, so that both images can be obtained in a short and simple circuit configuration. Both changes between data can be detected.
[0070]
  Further, in the present embodiment, the image comparison unit 7 is configured by a logic circuit (hardware), but is not limited thereto, and may be configured by software. In this case, the addition of all pixel data PIX by the adder 704 and the latch 705, the retention of the evaluation values ESTA / ESTB by the image latches 706 and 707, the subtraction between the two evaluation values ESTA / ESTB by the subtractor 708, and the subtraction result and a predetermined result The comparison with the value is replaced with software processing. In such a configuration, a processor such as the CPU 23 executes the comparison processing program to perform the above processing on the input pixel data PIX. this
In the processing, evaluation values ESTA / ESTB are held in a DRAM or the like.
[0071]
  As described above, when performing comparison processing with software, it takes more time than processing with hardware, so if you need to perform processing close to real time for operations that you do not know when to perform, It is desirable to adopt the configuration. However, if the arithmetic processing speed of the processor is sufficiently high, there is a possibility that the software can ensure the same level of immediacy as hardware.
[0072]
  Furthermore, in the present embodiment, the input pixel data PIX for one screen is stored in the FIFO memory 10 and then written in the image memory 5. However, the pixel data PIX is written in such a method. It is not limited to. For example, even if the write control signal WRITE does not become high level, the input image data is sequentially written in different memory areas of the image memory 5, unnecessary image data after writing is discarded, and the necessary image data is discarded. An example is a configuration that stores only data.
[0073]
  Specifically, only the image data when the write control signal WRITE is at a high level (when it is determined that the amount of change from the previously stored image data is large) is stored as it is, and the write control signal WRITE is low. When it is level (when it is determined that the amount of change is small), the image data is deleted. Therefore, the next new image data is written in the memory area from which the image data has been erased. In order to realize such a method, the address for writing the pixel data PIX to the image memory 5 is updated for each input image data, and the address of the memory area where the written image data is erased. Is used for writing the next image data without updating only. For example, when the image memory 5 is a DRAM, the image data is erased by controlling the operation of a refresh address counter that designates a row to be refreshed so that only the memory area to be erased is not refreshed. .
[0074]
  In such a configuration, the FIFO memory 10 for writing image data for one screen is not necessary, and the price of the operation history device can be reduced.
[0075]
【The invention's effect】
  As described above, the operation history device of the present invention includes a time generation unit that generates time, an image reading unit that reads an image of an operator who has performed an operation on the operation device, and an operation performed by the operator on the operation device. Operation detecting means for detecting information about the image, image storage means for storing the image read by the image reading means, operation information detected by the operation detecting means, and the time when the operator performs an operation Operation information storage means for storing the time generated by the generation means together with related information related to the image stored in the image storage means;A change amount determination for determining whether or not a change amount of the read image with respect to the stored image is larger than a predetermined value by comparing the image previously stored in the image storage unit and the image read by the image reading unit And a storage permission unit that permits storage of the read image in the image storage unit only when it is determined that the amount of change is greater than a predetermined value and the operation information is detected.It is the structure equipped with.
[0076]
  Thereby, the image memorize | stored in the image memory | storage means is linked | related by operation information and time by related information. As a result, the details of the operation performed and the time when the operation was performed can be easily specified together with the image of the operator. Therefore, the specific work of both can be simplified and the cause of the abnormality can be verified early.The Further, a change amount for determining whether or not a change amount of the read image with respect to the stored image is larger than a predetermined value by comparing the image previously stored in the image storage unit with the image read by the image reading unit. A determination unit; and a storage permission unit that permits storage of the read image in the image storage unit only when it is determined that the amount of change is greater than a predetermined value and the operation information is detected. If the operator does not change and the change in the read image with respect to the stored image is small, the read image is not stored, so that an extra image with a small change is not stored. Therefore, it is possible to reduce the storage capacity of the image and efficiently save the history of the operator and the operation content.
[0077]
  The operation history device further includes a synthesizing unit that synthesizes the image stored in the image storage unit with the operation information and time stored in the operation information storage unit. The operator, the operation content, and the operation time can be specified only by looking. Therefore, there is an effect that these specific operations can be performed more efficiently.The
[0078]
  UpIn the operation history device described above, the change amount determination means compares the two images with the sum of the data of all the pixels of the image, so that the comparison object is simplified and the comparison process can be easily performed. Accordingly, the efficiency of the comparison process is improved, and the effect of improving the immediacy of the comparison process with respect to an operation that is not known when it is performed can be achieved.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an operation history apparatus according to an embodiment of the present invention.
FIG. 2 is an explanatory diagram showing a memory map of an image memory and a history memory in the operation history device.
FIGS. 3A and 3B are explanatory diagrams for explaining a comparison method of an image comparison unit in the operation history apparatus.
FIG. 4 is a logic circuit diagram showing a configuration of a main part of the operation history device.
FIGS. 5A and 5B are timing charts showing signals used in the operation history device. FIG.
[Explanation of symbols]
  1 Video camera (image reading means)
  3 Key operation detection unit (operation detection means)
  4 RTC time stamp generator (time generator)
  5 Image memory (image storage means)
  6 History memory (operation information storage means)
  7 Image comparison unit (change amount judging means)
  8 AND gate (memory permission means)
11 Image composition unit (composition means)
21 Operation unit (operation device)

Claims (3)

Time generating means for generating time;
Image reading means for reading an image of an operator who has performed an operation on the operation device;
Operation detecting means for detecting information about an operation performed by an operator on the operating device;
Image storage means for storing an image read by the image reading means, operation information detected by the operation detection means, and time generated by the time generation means when an operator performs an operation, the image storage means an operation information storage means for storing together with related information related to the stored image,
Change amount determination means for determining whether or not the change amount of the read image with respect to the stored image is larger than a predetermined value by comparing the image stored in the image storage means with the image read by the image reading means. When,
An operation comprising: a storage permitting means for permitting storage of the read image in the image storage means only when the change amount is determined to be greater than a predetermined value and the operation information is detected. History device.   2. The operation history apparatus according to claim 1, further comprising a combining unit that combines the image stored in the image storage unit with the operation information and time stored in the operation information storage unit. The operation history apparatus according to claim 1, wherein the change amount determination unit compares the two images with a sum of data of all pixels of the image .

Images