CN103428433B

CN103428433B - Image processing device, image processing system and image processing method

Info

Publication number: CN103428433B
Application number: CN201310198350.9A
Authority: CN
Inventors: 王浩
Original assignee: Hangzhou Mission Infrared Electro Optics Technology Co Ltd
Current assignee: Hangzhou Mission Infrared Electro Optics Technology Co Ltd
Priority date: 2012-05-23
Filing date: 2013-05-23
Publication date: 2024-04-16
Anticipated expiration: 2033-05-23
Also published as: CN103428433A

Abstract

The invention relates to an image processing device, an image processing system and an image processing method, and relates to the application fields of image processing devices, image processing systems and image shooting. In the prior art, the shot object to be shot is selected by depending on subjective experience of a user, so that the efficiency is low and omission is easy. The invention provides an image processing device, an image processing system and an image processing method, wherein object information and the ordering order of the object information are determined based on pre-stored object information, then special object information is specified, object indication information obtained by the special object information is particularly displayed as the information prompt of the current object to be shot, and when a switching instruction is sent, the special object information is switched and specified according to the ordering order. Thus, the shooting speed is increased, and omission is not easy to occur.

Description

Image processing device, image processing system and image processing method

Technical Field

The invention relates to an image processing device, an image processing system and an image processing method, and relates to the application fields of image processing devices, image processing systems and image shooting.

Background

As a well-known technology, various photographing devices such as infrared, visible light, ultraviolet, laser imaging and the like are widely used in fields such as industry, and for example, photographing of electric power equipment, buildings and the like at regular intervals is an important part of state maintenance.

The image shooting workload is quite heavy, taking a 500KV power transformer substation as an example, the transformer substation generally has dozens of equipment areas, each equipment area also has dozens of objects to be shot, and in order to reduce the labor intensity of users, an image processing system formed by connecting an image processing device and an image shooting device is used for carrying on a moving platform such as a vehicle to carry out inspection shooting on the objects. The current technology, shooting personnel need select the shot object that needs to shoot according to subjective experience, detection efficiency is low, working strength is big, still easily mistakes and miss the shot object, and its reason is: the number of the shot objects is large, and the shot objects of the same type are large, and sometimes shooting is required from multiple angles, so that missed detection and false detection are easy. At present, no suitable means is available for conveniently realizing information prompt of an object to be photographed, and a user is often confused in photographing.

Since the application of image capturing technology, those skilled in the art have been trying to solve the above problems, and some means for reducing the technical difficulty of image capturing and improving the capturing speed are known. For example, patent document application No.: 200410001328 discloses a camera which can be used to notify information of an identified object by receiving a wireless tag ID attached to the object, but the wireless tag is inconvenient to attach, increases in cost, and is not applicable in some fields where electromagnetic interference is large; how to instruct a user to take a photograph is a difficult problem.

Accordingly, it is appreciated that there is a need for an image processing apparatus that can provide a user with object prompt information for photographing an object to perform photographing of the object, and solve the drawbacks caused by personal habit photographing, thereby solving one or all of the problems described above.

Further, when the characters of the subject indication information are more, how to facilitate the observation of the user is also an optimization consideration problem; when the number of the shot object information is large, how to quickly call the display without occupying the operation resource of the processor is also an optimization consideration problem; after the object information is recorded in association with the image file, how to facilitate the computer to quickly read the object information is also an optimization problem.

In addition, in some application occasions, many shot objects have specific shooting positions, shooting angles and shooting distances, but currently, shooting staff still need to select the shooting positions of the shot objects by experience and habit, so that the shooting speed is low, and key measurement positions are easy to miss. And if the position, size and angle of the similar object images shot each time are large, the subsequent batch processing can be difficult; therefore, how to normalize the quality of the shot is also an issue that can be optimally considered.

Disclosure of Invention

The invention provides an image processing device, an image processing system and an image processing method, wherein a plurality of pieces of object information related to an object to be shot are stored in advance, and the object information related to the object to be shot and the ordering order in shooting are determined based on the stored object information during shooting; the method comprises the steps of designating special object information in the special object information, displaying the image and the object indication information obtained by the special object information in a special display mode, enabling a user to use the special object information as an information prompt of an object to be shot currently, and distinguishing and checking the object according to the knowledge of the object on site, and then shooting the object; and then, the special object information is switched and designated according to the sorting order in response to the switching instruction operation, and the object instruction information obtained by switching the designated special object information is particularly displayed, wherein the switching instruction operation is sent by a user pressing a switching key, so that the object information is not required to be searched and selected, and the operation is simple. Thus, the aim of prompting the user of the shot object which is required to be shot currently according to the preset sorting order is achieved. Obviously, the shooting speed is improved, and omission is not easy to occur.

Therefore, the invention adopts the following technical scheme that the image processing device comprises:

a communication unit configured to continuously receive video data;

an information specifying section for specifying, based on the plurality of pieces of subject information stored in the information storage section, subject information among them as particular subject information;

a display control unit configured to control the display unit to display the image generated by the image data continuously, and to display, in a special display manner, the object instruction information obtained from the special object information specified by the information specification unit;

The information designating unit is configured to switch the designated specific subject information in the order of the subject information based on the subject information stored in the information storage unit in response to a switching instruction operation or in accordance with a predetermined switching condition.

In another aspect, an image processing apparatus includes:

a communication unit configured to continuously receive video data;

A display control unit configured to control the display unit to display a moving image generated from the image data, and to display, in particular, subject instruction information obtained from the particular subject information specified by the information specification unit, and/or to display a reference image obtained based on reference image configuration data associated with the particular subject information;

The information designating unit switches the designated specific subject information based on the order of the subject information in response to a switching instruction operation or in accordance with a predetermined switching condition.

In still another aspect, an image processing apparatus includes:

a communication unit configured to continuously receive video data;

An information specifying section for specifying reference image composition data for obtaining a reference image based on the reference image composition data stored in the information storage section;

a display control unit configured to control the display unit to display a dynamic video generated from the video data and to display a reference image obtained based on the specified reference image configuration data;

The information designating unit switches the designated reference image configuration data based on the order of the reference image configuration data in response to a switching instruction operation or in accordance with a predetermined switching condition; the display control unit is configured to control the display unit to display the video generated by the video data and to display the reference image obtained by switching the specified reference image configuration data according to the information specification unit.

The image processing method of the invention comprises the following steps:

A communication step for continuously receiving image data;

an information specifying step of specifying, based on the plurality of pieces of subject information stored in the information storage section, subject information among them as particular subject information;

a display control step of controlling a display section to display, in a special display manner, the object instruction information obtained from the special object information specified in the information specifying step while causing the display section to continuously display the image generated from the image data;

The information specifying step switches the specified special subject information in the order of the subject information based on the subject information stored in the information storage section in response to a switching instruction operation or in accordance with a prescribed switching condition.

Additional aspects and advantages of the invention will be set forth in the description which follows.

Description of the drawings:

fig. 1 is an electrical block diagram of an image photographing device according to an embodiment.

Fig. 2 is an exterior view of an image photographing device according to an embodiment.

Fig. 3 is a diagram showing a display interface of a task and order setting menu.

Fig. 4 is a control flow diagram showing one example of the information pattern.

Fig. 5 is a schematic list example of object information stored in the information storage section.

FIG. 6 is a schematic diagram of a setup interface of one example of a sort setup process.

Fig. 7 is an example of a ranking factor corresponding to object information obtained after ranking setting.

Fig. 8 is a display interface diagram of an example before and after switching of the subject instruction information, which is particularly displayed.

Fig. 9 is a display interface diagram of one example of the subject instruction information that is particularly displayed.

Fig. 10 is a display interface diagram of one example of the subject instruction information particularly displayed.

Fig. 11 is a schematic diagram showing the distribution of the subjects in the device areas 1,2, 3 in the substation 1 and the photographing route in the sorting order corresponding to the subjects according to embodiment 1.

Fig. 12 is a control flow diagram showing another example of the information pattern.

Fig. 13 is a schematic diagram of a setting interface of one example of the filter conditions.

Fig. 14 is a schematic diagram of a setting interface of another example of the filter conditions.

Fig. 15 is another example of a schematic list of object information stored in the information storage section.

Fig. 16 is an example of a schematic list of object information determined according to the filter conditions.

Fig. 17 is a diagram of a display interface before and after adjustment of the ranking order.

Fig. 18 is a schematic diagram of an example of a display interface in which the display section simultaneously displays subject instruction information and a plurality of types of reference images.

Fig. 19 is a schematic diagram of one example of subject information and associated reference image configuration data stored in the information storage section.

Fig. 20 is a flowchart showing another control embodiment of the information pattern.

Fig. 21 is a schematic diagram of an example of one display interface before and after shooting and switching of a reference image, which is specifically displayed object instruction information.

Fig. 22 is a schematic diagram of an example of a data format of a recorded video file.

Fig. 23 is a schematic diagram of one example of reference image configuration data stored in the information storage section of embodiment 4.

Fig. 24 is a flowchart showing another control embodiment of the information pattern.

Fig. 25 is a display interface example of the reference image of embodiment 4.

Fig. 26 is another display interface example of the reference image of embodiment 4.

Detailed Description

Fig. 1 is a block diagram showing an electrical configuration of an embodiment of an image processing system configured by connecting an image processing apparatus 100 and an image capturing apparatus 101. Fig. 2 is a schematic diagram of an implementation of an image processing system in which the image processing apparatus 100 and the image capturing apparatus 101 are connected.

In embodiment 1, the image capturing apparatus 101 is connected to the image processing apparatus 100 by a cradle head or the like mounted on the detection vehicle via a communication line such as a dedicated cable or a local area network configured by wired or wireless systems. The user views and monitors the subject image by the image processing apparatus 100. The image capturing device 101 is connected to the image processing device 100 to form an image processing system in the embodiment, and is configured to capture an object to obtain image data and output the image data.

The image capturing device 101 may be various types of image capturing devices, such as a thermal imaging device, a visible light imaging device, an ultraviolet imaging device, a laser imaging device, and the like, which are used to capture an object and output image data.

Specifically, an embodiment is illustrated, and an electrical block diagram of the image capturing apparatus 101 in fig. 1 is configured by a communication interface 10, a capturing unit 20, a flash memory 30, a temporary storage unit 50, an image processing unit 40, a CPU60, and the like.

The CPU60 controls the overall operation of the imaging device 101, and the flash memory 30 stores a control program and various data used for controlling the respective parts. The photographing part 20 includes an optical component, not shown, composed of an optical lens, for focusing the received light to the image sensor, a driving component, an image sensor, and a signal preprocessing circuit; the driving part drives the lens to perform focusing or zooming operation according to a control signal of the CPU60, and may be an optical part that is manually adjusted; an image sensor such as a CMOS sensor that converts light passing through the optical member into an electrical signal; the signal preprocessing circuit performs signal processing such as sampling and automatic gain control on the signal read from the image sensor at a predetermined period, and converts the signal into a digital image signal through the AD conversion circuit; the image signal is temporarily stored in the temporary storage unit 50, and then subjected to predetermined processing (e.g., compression processing) by the image processing unit 40 (e.g., DSP) to obtain image data for output, and the obtained image data is output via the communication interface 10. Depending on the design and purpose of use, for example, the image data output from the image capturing device 101 may be an original or predetermined processed image signal, or may be data obtained after predetermined processing of an image signal, for example, one or more of image data of an image (image data of an image generated from an image signal), compressed data of an image signal or an image data of an image in a predetermined format, and the like, which are collectively referred to as image data.

When the photographing device 101 is various different types of imaging devices, there are differences in optical components, image sensors, and the like. If the photographing device 101 is a thermal imaging device, the optical component is an infrared optical component, and the image sensor is an infrared detector such as a refrigerating or non-refrigerating infrared focal plane detector; when image data of an image (infrared thermal image) is to be obtained, the image processing section 40 is configured to process the image signal (thermal image signal) such as pseudo color processing, for example, to determine a corresponding pseudo color plate range from a range of AD values or a set range of AD values of the image signal (thermal image signal), and to use a specific color value corresponding to the AD value of the image signal (thermal image signal) in the pseudo color plate range as image data of a corresponding pixel position in the infrared thermal image. In one embodiment, the thermal image data (image data) output by the photographing device 101 may be thermal image signals after predetermined processing, or may be data obtained after predetermined processing of thermal image signals, for example, one or more of temperature value array data of thermal images, image data of infrared thermal images (image data of infrared thermal images generated by thermal image signals), data compressed in a predetermined format, and the like, and collectively referred to as thermal image data.

When the photographing device 101 is a visible light photographing device, the optical component is a visible light optical component, and the image sensor is a visible light CMOS sensor; when image data of a video (visible light) is to be obtained, the image processing unit 40 performs predetermined processing on the video signal, for example, various image processing such as white balance compensation processing, Y compensation processing, YC conversion processing, and the like, and generates visible light image data composed of a digitized luminance signal and a color difference signal.

Here, the image capturing device 101 is used for capturing and outputting image data, and the above-described differences do not cause significant differences in the following embodiments. The ideas of the following embodiments are applicable to a general image capturing device, such as various types of imaging devices including a visible light capturing device, a thermal imaging device, an ultraviolet capturing device, a laser imaging device, and the like; or a device comprising more than one shooting function formed by the combination of the devices, that is, the shooting device 101 may have a plurality of similar or different image sensors.

The image processing apparatus 100 of the present embodiment includes a communication interface 1, an auxiliary storage unit 2, a display control unit 3, a display unit 4, a hard disk 5, a temporary storage unit 6, an operation unit 7, and a CPU8 connected to the above components via a bus and performing overall control. The image processing apparatus 100 may be a computer or a dedicated processing apparatus, for example. The image processing apparatus 100 receives image data output from the image capturing apparatus 101 connected to the image processing apparatus 100 through the communication interface 1 based on the control of the CPU8.

A communication interface 1 for continuously receiving image data output from the image capturing device 101; wherein the method includes receiving video data transmitted by a relay device (video data output by the video capturing device 101 is transmitted by the relay device); meanwhile, the device can also be used as a communication interface for controlling the image shooting device 101. Here, the communication interface 1 includes various wired or wireless communication interfaces on the image processing apparatus 100, such as a network interface, a USB interface, a 1394 interface, a video interface, and the like, and is configured with a wireless communication unit, such as GPRS, 3G, 4G, 5G, and a wireless network card (e.g., a WIFI unit), for wirelessly receiving image data. The communication interface 1 is an example of a communication section.

The auxiliary storage unit 2 is a storage medium such as a CD-ROM or a memory card, and an associated interface.

The display control unit 3 performs display of the image data for display stored in the temporary storage unit 6 on the display unit 4 in accordance with control by the CPU 8. For example, in the present embodiment, in the shooting standby mode, images generated from image data obtained by shooting are continuously displayed; in the information mode, the specified (including at least the specifically displayed) subject instruction information and the video (including the dynamic video and the still video) are simultaneously displayed, and in the playback mode, the video read out and expanded from the hard disk 5 is displayed, and various setting information can be displayed. Specifically, in one embodiment, the display control unit 3 includes a VRAM, a VRAM control means, a signal generation means, etc., and based on the control of the CPU8, the signal generation means periodically reads out the image data read out from the temporary storage unit 6 and stored in the VRAM through the VRAM control means, generates a display signal, for example, a video signal output, and displays the display signal on the display unit 4; the display control unit 3 may be integrated with the CPU 8.

The display portion 4 is, for example, a liquid crystal display, and in the present embodiment, the display portion 4 is exemplified as the display portion. The display unit 4 is not limited to this, and may be another display connected to the image processing apparatus 100, and the image processing apparatus 100 itself may have no display in its electrical structure. Obviously, when the image processing apparatus 100 itself has no display device in its electrical structure, the CPU8 may control the output of image data for display, for example, through the communication interface 1 (for example, various wired or wireless image output interfaces such as an AV port, an RJ45 port, etc.), and output the image data for display (or it may be understood that the CPU8, the display control section 3, etc. are examples of the display control section); the display control unit controls the display unit to display the image, and also includes the display output.

The hard disk 5 stores therein a program for control and various data used in the control.

The temporary storage unit 6 is a volatile memory such as a RAM or a DRAM, and is a buffer memory for temporarily storing video data received by the communication interface 1. Meanwhile, functioning as a work memory of the CPU8, data processed by the CPU8 is temporarily stored. The memory, the register, and the like included in the processor such as the CPU8 may be interpreted as a temporary storage unit.

An information storage section for storing a plurality of pieces of subject information in advance. Here, the information storage unit refers to both a storage medium in the image processing apparatus 100, such as a nonvolatile storage medium such as the hard disk 5, and a volatile storage medium such as the temporary storage unit 6; other storage media connected to the image processing apparatus 100 by wire or wireless, such as other storage devices communicating via wire or wireless connection to the communication interface 1, or storage media in other imaging devices, computers, and the like, may be used. Preferably, the subject information is stored in advance in the image processing apparatus 100 or in a nonvolatile storage medium connected thereto.

The subject information is information about a subject (to be photographed), for example, information representing a subject location, type, number, or the like, and various kinds of information about a subject such as a attribution unit, a classification level (e.g., a voltage level, an importance level, or the like), a model number, a manufacturer, performance, and characteristics, a history of past photographing or inspection, a date of manufacture, a lifetime, or the like may be mentioned.

In a preferred embodiment, various pieces of information in the subject information are configured in the form of classification information, and an information storage unit shown in an exemplary list 5001 in fig. 5 stores one implementation schematic diagram of the subject information, each of which is configured of attribute information of a plurality of prescribed attributes, such as subject information 500 "substation 1 device zone 1 device ia phase" having attribute information "substation 1" corresponding to the substation attribute 501, attribute information "device zone 1" corresponding to the device zone attribute 502, attribute information "device i" corresponding to the device type attribute 503, and attribute information "a phase" corresponding to the phase attribute 504. However, the present invention is not limited to this, and the subject information may be stored in a form other than the form of information classification.

An operation unit 7: for the user to perform a switching instruction operation, record an instruction operation, or input various operations such as setting information, the CPU8 executes a corresponding program based on an operation signal of the operation unit 7, and the operation unit 7 is configured by, for example, a keyboard, a mouse, or the like, and the operation may be realized by using a touch panel, a voice control unit, or the like, without being limited thereto.

The CPU8 controls the overall operation of the image processing apparatus 100, and the hard disk 5 stores therein a program for control and various data used for control of each part in the hard disk 5. The CPU8 controls the transfer of the image data received through the communication interface 1 to the temporary storage section 6; the CPU8 performs predetermined processing such as correction, interpolation, pseudo-color, synthesis, compression, decompression, and the like, and performs processing for converting the data into data suitable for display, recording, and the like.

The CPU8 performs predetermined processing on the received video data to obtain image data of a video. According to different formats of the image data, for example, when the received image data is a compressed image signal, a predetermined process such as the CPU8 decompresses the image data received by the communication interface 1 and performs a corresponding predetermined process; for example, when the image capturing apparatus 101 is a thermal imaging apparatus, in one embodiment, the output is a compressed infrared image signal (image data), and after decompression, a predetermined process such as a pseudo color process is performed, for example, a corresponding pseudo color plate range is determined from an AD value range of the infrared image signal or a set range of AD values, and specific color values corresponding to each AD value in the pseudo color plate range are used as image data corresponding to pixel positions in an image to obtain image data of the image (infrared image), and predetermined processes such as correction, interpolation, and other predetermined processes are performed on the decompressed image data. In another embodiment, for example, when the received image data itself is already the image data of the compressed image, the image data of the image is decompressed to obtain the image data of the image. In still another embodiment, for example, when the communication interface 1 receives an analog video, control is performed to transmit the image data of the digital video obtained after AD conversion by the relevant AD conversion circuit to the temporary storage unit 6.

In the following, an image processing method (information mode) will be described, and in the present embodiment, the CPU8 serves as an example of the order determining section, the information specifying section, and the display control section. In the information mode of the image processing apparatus of the present embodiment, taking a scene in which a user performs image capturing on an object whose device type is device i, device ii, or device iii in device area 1 (1101), device area 2 (1102), or device area 3 (1103) in substation 1 shown in fig. 11 as an example; prior to the main shooting, the hard disk 5 stores therein subject information corresponding to these subjects in advance in the form of a database, a data file, or the like.

In the vehicle shown in fig. 2, which includes the image processing apparatus 100, the image capturing apparatus 101, and necessary power supply, communication cable, etc., after the image processing apparatus 100 and the image capturing apparatus 101 are connected and started, the CPU8 performs control and related processing thereof, and causes the image processing apparatus 100 to receive the image data output from the image capturing apparatus 101 and enter a standby display mode, and in one embodiment, the image data received by the communication interface 1 is transferred to the temporary storage unit 6 at a predetermined frame rate, and subjected to predetermined processing to obtain the image data of the image, and stored in the temporary storage unit 6, and causes the display unit 4 to continuously display the image in the form of a moving image, and in this state, the CPU8 performs control thereof, continuously monitors whether the processing is switched to another mode according to a predetermined condition or the shutdown operation is performed, and if so, enters corresponding processing control. Referring to fig. 3, the control procedure of the information mode is as follows:

in step S101, the CPU8 performs control to continuously monitor whether or not the user has selected the information mode through the operation unit 7, and when it is detected that the information mode has been selected, the flow advances to step S102.

Step S102, an ordering factor of object information is set.

The CPU8 controls to cause the display section 4 to display a setting interface as shown in fig. 3, having a plurality of setting items such as a group setting item 301 configured to perform a group sorting setting operation, a sorting factor item 302 to perform an object information sorting factor setting operation, a filter condition item 303 to perform a filter condition setting operation, and a determination item 300 to determine settings; and then continuously detects whether the user selects one of the setting items for the setting operation or makes a determination of the determination item 300. Entering the corresponding setting interface if one of the setting items is selected, returning to the setting interface shown in fig. 3 after the corresponding setting is completed and determined, the CPU8 continues to detect whether the user selects one of the setting items for the setting operation or the determination of the determination item 300; then, when the determination of the determination item 300 is detected, the process advances to step S103; wherein if the determination item 300 is determined without performing the operation of other settings, the process proceeds to step S103 according to the previous default settings.

In this embodiment, the user selects the information ordering factor setting item 302; next, in response to this operation, the CPU8 causes the display section 4 to display a sort setting interface 6000 as shown in fig. 6, and the user sets the sort factors of the object information against the substation layout diagram as shown in fig. 11. In the present embodiment, the display section 4, the operation section 7, and the CPU8 constitute an example of a sort setting section.

The CPU8 transfers the subject information stored in the hard disk 5 to the temporary storage unit 6, and the transferred subject information may be all attribute information or predetermined partial attribute information; further, when the power is turned on, the subject information may be transferred to the temporary storage 6.

A ranking setting section for a user to set ranking factors of the subject information; the ranking setting section has a keyword determination unit for determining keywords related to the subject ranking; a ranking factor determining unit for determining ranking factors of the keywords; the ranking setting section is configured to combine the first keyword ranking factor or the first keyword ranking factor with the auxiliary keyword ranking factor to obtain a ranking factor of the subject information. The ranking factor refers to a ranking order or a ranking rule, and the ranking rule is, for example, a numerical ranking rule, a letter ranking rule, a Chinese pinyin ranking rule, a Chinese stroke ranking rule, a ranking rule obtained by manually ranking keywords by a user, and the like.

Wherein when the subject information participating in the ranking is subject information having different first keywords, the ranking setting section is configured to set the first keyword ranking factors as ranking factors of the subject information. When a plurality of pieces of object information with the same first keywords are included in the object information participating in the sorting, the sorting factors of the object information are determined by the combination of the first keyword sorting factors and the auxiliary keyword sorting factors. Keywords that are first ranking elements are referred to as first keywords; the auxiliary keyword ranking factors include at least ranking factors of the second keywords, and the keywords in the subject information as the second ranking elements are called second keywords. When a plurality of pieces of subject information having the same first keyword and second keyword are provided, the auxiliary keyword ranking factor includes at least the second keyword and the third keyword ranking factor, and a keyword as a third ranking factor in the subject information is referred to as a third keyword; and so on.

A keyword determination unit configured to determine keywords related to the ranking; for example, keywords related to ranking may be determined in one or more of the following implementations. Wherein the same or different determination embodiments may be employed when having the first keyword and the auxiliary keyword.

In one embodiment, a keyword of attribute information of a predetermined attribute in object information is searched for based on the object information stored in the information storage section, and the keyword obtained by the search is determined as a keyword related to the ranking;

Further, it is also possible to query, based on the subject information and the keyword query condition stored in the information storage section, keywords of attribute information of a prescribed attribute in the subject information conforming to the keyword query condition, and to determine the keywords obtained by the query as keywords related to the ranking;

in addition, keywords can be pre-stored for a user to select, and then the keywords selected by the user are determined to be keywords related to the ordering;

In addition, by determining the keyword attribute, the keywords conforming to the keyword attribute can be determined as the keywords ordered; for example, when the subject information is associated with time information (for example, the data file is a log file containing the subject information and time), providing a keyword attribute (such as a time attribute, etc.) for selection, and determining a keyword (time) conforming to the attribute as a keyword in order based on the keyword attribute (such as the time attribute) selected by the user;

In addition, a default keyword may also be determined as a ranking related keyword;

In addition, keywords entered by the user may also be determined as keywords related to the ranking.

A ranking factor determining unit for determining ranking factors of the keywords; for example, the ranking factor may be determined in one or more of the following embodiments; wherein the same or different determination embodiments may be employed when having the first keyword and the auxiliary keyword.

In one embodiment, pre-storing ordering factors for a user to select, and determining the ordering factors selected by the user as ordering factors corresponding to the keywords; further, for example, a default ranking factor; in addition, ranking factors for the keywords are obtained based on the ranking operations of the user (including the ranking factors entered by the user).

Specifically, in the present embodiment, based on the subject information stored in the information storage section, the first keyword of the attribute information of the prescribed attribute in the subject information is queried and the ranking factor of the first keyword is determined based on the operation of the user. The CPU8 causes the display section 4 to display a sort setting interface 6000 as shown in fig. 6, in which the device region setting item 601, the device type setting item 602, and the phase setting item 603 are menu items configured to represent keyword query and sort factor settings; as shown in the order setting interface 6001, when the (or default) device region setting item 601 is selected, the CPU8 inquires of the keywords of the attribute information of the device region attribute of the subject information, obtaining a device region keyword (first keyword) list 605; the ranking factor of the first keywords is then determined based on the ranking order between the first keywords (device region 1, device region 2, device region 3) set by the user. The order in the order list 604 may be a number entered, or may be a keyword in the device area keyword list 605 selected, and the order may be set by drag operation through a cross key or a touch screen.

In this embodiment, the key of the attribute information of the device area is used as the first key, and the ranking factor of the object information needs to be determined by the combination of the first key ranking factor and the auxiliary key ranking factor among the object information participating in the ranking, which has the same first key.

A keyword determination unit configured to query, based on the subject information stored in the information storage unit and the determined keyword query condition, an auxiliary keyword of attribute information of a prescribed attribute in the subject information conforming to the keyword query condition; wherein the keyword query condition is one of the first keywords determined before, and when the auxiliary keywords above the second keywords are provided, the keyword query condition is one of the first keywords determined before and one of the keywords of each level of auxiliary keywords of each level; but not limited to, the keyword query condition can also be input.

Further configured such that, when the user selects the device area 1, the cpu8 determines the device area 1 (one of the first keywords, the underlined keyword in the device area keyword list 605 in the sort setting interface 6002) as the keyword query condition; the CPU8 obtains a device type attribute information keyword (second keyword) by querying device type attribute information of subject information conforming to the "device region 1" keyword query condition, displays a device type keyword list 607, and the user sets the ranking of the device types therein to obtain the ranking order in the ranking list 606; thereafter, the CPU8 uses "device i" selected by the user as one of the keyword query conditions (the keyword indicated by the underline in the device type keyword list 607 in the ranking setting interface 6003), and the CPU8 obtains the device-related keyword list 609 by querying the device-related attribute information of the subject information conforming to the "device area 1" and "device i" keyword query conditions, sets the ranking of the other therein, and obtains the ranking order in the ranking list 608. Then, the device types and the ranks of the device phases in the device area 1, the device area 2, and the device area 3 are set in a similar manner; when the setting of the sorting factors is completed, the sorting settings are determined by pressing down the determination key 4 of the operation section 7, returning to the setting interface shown in fig. 3, and then determining the determination item 300 by pressing down the determination key 4 of the operation section 7 again; the ranking factor corresponding to the subject information shown in the schematic list of fig. 7 is obtained, and the set ranking factor of the subject information is recorded in the temporary storage section 6, or in the hard disk 5 or the like, for example, a configuration file of the ranking rule is generated, or a data file of the subject information to which the ranking order is added is generated, and the flow advances to step S103.

Note that, in the present embodiment, the ordered first keywords are queried based on the subject information stored in the information storage section; but as a filtering condition for determining the subject information is set (or has a default) before, it may be configured that the filtering condition is one of the keyword query conditions of the keywords (including the first keyword and the auxiliary keyword) described above.

The method has the advantages that the inconvenience of adjusting the sequence caused by displaying a large amount of shot object information and the corresponding sequencing sequence on the display part is avoided, and the sequencing keywords obtained by query are further simplified and the inconvenience of sequencing setting caused by displaying redundant information is avoided through the keyword query condition.

In addition, when the information storage part stores a plurality of groups of object information, a sorting factor is set between groups, and a sorting factor between data files is set; when the user selects the group setting item 301 shown in fig. 3 therefrom, the list of data files is caused to be displayed based on the data files (groups) stored in the information storage section, and the user sets the sort factor for the data files in the list; and a setting part for setting the ordering factors among the groups. The group herein refers to a collection containing a plurality of subject information, such as a data file, a folder, etc., having a common group identification; in addition, after the group sorting is completed, the same group of object information can be set or checked again.

Step S103, determining the sorting order of the object information.

A sorting determination section for determining a sorting order for the subject information stored in the information storage section. When the ranking factor is a ranking rule, the ranking determining section is configured to determine a ranking order of the subject information by, for example, extracting keywords and ranking the subject information according to the ranking rule of the keywords. When a plurality of keywords exist, sorting is performed according to the first keywords and rules, then sorting is performed on the object information with the same first keywords according to the second keywords, and so on.

In this example, since the sorting factor set in step S102 is the sorting order of the attribute information based on the subject information stored in the information storage unit, the sorting of the subject information may be performed according to the sorting order of the attribute information, and the sorting order of the subject information may be determined; if the data file to which the subject information of the sort order is attached is directly generated in step S102, the subject information may not need to be sorted.

In addition, the step of setting the ranking factor (S102) is not necessary, and the ranking order may be determined according to a pre-stored ranking factor such as a default ranking factor. In one embodiment, the information storage unit (e.g., the hard disk 5) stores therein the subject information to which the sorting order information is added, and the sorting determination unit determines the sorting order of the subject information to be followed at the time of the subsequent switching based on the sorting order information to which the subject information is added; the additional ordering information is identifiable ordering information, such as number, letter, code, etc.; may be attached as a form of attribute information of the subject information; or may be attached as a form of index information corresponding to the subject information; alternatively, the subject information may be stored at a predetermined address in a data file or a storage medium, or may be regarded as a form to which ordering information is added.

The ordering of the subject information may also be determined by ordering the subject information according to a default ordering rule (e.g., according to a temporal attribute of the subject information). The display unit may display object instruction information generated from a predetermined number of pieces of object information, and the sorting order may be determined so that the object instruction information is sorted by a cross key or a touch panel and drag operation.

Step S104 designates object information as special object information. An information specification section (CPU 8) for specifying, based on the subject information stored in the information storage section, subject information as special subject information for obtaining subject instruction information for special display; in a preferred embodiment, the object information "substation 1 equipment area 1 equipment ic phase" first in the sorting order is designated as the special object information based on the sorting order.

In other embodiments, the CPU8 monitors whether or not there is an operation instruction to select particular subject information; here, in the case where the operation instruction is confirmed, the selected subject information is designated as the special subject information; for example, among the object instruction information shown in the object information display field 801 of fig. 8, the user can select one of the object instruction information through the operation section 7, and the CPU8 designates the object information corresponding to the object instruction information as the special object information.

Step S105, transferring the image data received through the communication interface 1 to the temporary storage 6; the CPU8 performs predetermined processing on the video data to obtain image data of the video, and stores the image data in the temporary storage unit 6.

In step S106, the CPU8 controls the display unit 4 to display the video generated by the video data and to display the subject instruction information obtained by the particular subject information specified by the information specification unit.

Here, the object instruction information and the image are displayed together on the display interface, and may be superimposed on the image and/or may not be superimposed on the image.

The special display means that only the object instruction information obtained by the specified special object information is displayed, for example, as shown in fig. 9.

In particular, it also means that, when object instruction information obtained from a predetermined number of object information other than the particular object information is displayed, the object instruction information obtained from the specified particular object information is particularly displayed in a display manner different from other object instruction information. In a preferred embodiment, the display control unit controls the display unit to display the video generated from the video data based on the subject information stored in the information storage unit (determined when a task is determined) and the order of the subject information determined by the order determination unit, and to display the subject instruction information obtained by the predetermined number of subject information in order in the order, wherein the subject instruction information obtained from the specified specific subject information is displayed in a display manner different from the other subject instruction information.

As shown in a display interface 801 in fig. 8, the CPU8 controls to display the object indication information obtained by the predetermined number of object information in order, and the underline mark of the "substation 1 equipment area 1 equipment ic phase" is distinguished from other object indication information, and a difference in display position, color, background, size, font, text description, and the like, which are convenient for the user to distinguish, may be adopted as a special display form distinguished from other object indication information.

In a preferred embodiment, the display control unit performs layered display of predetermined information for generating object instruction information among the object information, the layered display including tree-like display, at predetermined positions. For example, an information display column 1001 shown in fig. 10, in which a display control section divides object information into three layers for display at predetermined positions according to attribute information representing a substation, an equipment area, an equipment type, and a corresponding predetermined attribute; the user can observe conveniently.

Specifically, in one embodiment of display control, the CPU8 synthesizes image data of a video generated by capturing the obtained video data, and image data of subject instruction information obtained by specifying particular subject information; storing the synthesized image data in the temporary storage unit 6; then, the control display control unit 3 displays the synthesized image data on the display unit 4. As shown in fig. 9, the composition is to superimpose the subject instruction information on the video. When there is any other predetermined instruction information, the CPU8 synthesizes the image data of the video generated by capturing the obtained video data, the image data of the specific display object instruction information obtained by the specific object information (when there is a specific display mark, the image data including the mark, such as the underline of the object instruction information in fig. 8), and the image data of the other predetermined instruction information; and controls the display control section 3 to display the synthesized image data on the display section 4, as shown in fig. 8, the synthesized image data is obtained by combining the information display field containing the subject instruction information and other instruction information with the video. Other instruction information includes a predetermined number of pieces of subject information, progress information, and a ranking order, and also includes instruction information such as a date and a battery capacity that the display unit 4 needs to display.

The object instruction information obtained from the object information may be obtained by the whole or predetermined part of the object information, and the configuration of the object information for obtaining the object instruction information may be predetermined.

Wherein the CPU8 serves as a progress statistics section for counting shooting progress information based on the number of pieces of subject information (determined subject information when there is a task determination, or the number of sequence numbers according to a sort sequence, or the like) stored in the information storage section and the specific subject information currently specified; progress information 804 as displayed in progress information 803 and 802 displayed in 801 in fig. 8; note that in fig. 8, in the case where a sorting order of the subject information specified has been displayed, the sorting order may also constitute progress information with only the total number of subject information "15" displayed.

The order of the object information may or may not be displayed, and order instruction information is displayed in fig. 8.

In step S107, the CPU8 determines whether or not a switching instruction operation is performed or whether or not a predetermined switching condition is satisfied, and if not, steps S105 to S107 are repeated to display the subject instruction information together with the moving image (continuous composition), and if the switching instruction operation is performed from the operation unit 7, the process proceeds to step S108.

The user checks and photographs the corresponding subject with the subject prompt message "substation 1 equipment area 1 equipment ic phase" specifically displayed as the prompt message for photographing the current subject.

When the user completes shooting of the corresponding object according to the object indication information "the device ic phase of the device area 1 of the substation 1" in the information display field 801 in fig. 8, and then presses the key representing the switch in the operation part 7, the switch indication operation is completed by the one-touch operation, and the user does not need to select or search for the object information shown in the information display field 801, so that the operation is simple.

Further, the CPU8 may be configured to perform the switching based on other predetermined switching conditions; for example, when it is determined that a predetermined time interval is satisfied, for example, when a criterion threshold is satisfied for image analysis (for example, when a temperature value in an infrared thermal image exceeds a predetermined threshold), for example, when a predetermined condition such as a trigger signal of another sensor device connected to the image processing apparatus 100 is detected, the process proceeds automatically to step S108 to execute a process of switching specification even if a switching instruction operation from the operation unit 7 is not performed. The predetermined switching conditions include processing for performing switching specification when other predetermined conditions are satisfied at the same time, for example, a predetermined time interval is satisfied or a trigger signal is received even if a switching instruction operation of the operation unit 7 is performed. Obviously, the prescribed switching condition may be various conditions configured in advance.

In step S108, the CPU8 determines whether the task is completed.

In response to the switch instruction operation, the CPU8 judges whether or not the specification of the last subject information in the sort order is completed; if not, the process returns to step S104, and at this time, the specified special subject information is switched in the order of the sorting. If the information mode is finished, words such as 'task completion' can be displayed on the screen, and then the information mode returns to the standby state of the infrared shooting device to wait for the operation of entering other modes by a user. Further, the imaging can be performed by switching to a predetermined order start point, for example, a start point of the order.

Specifically, in one embodiment, the information designating unit (CPU 8) adds 1 to the sequence number of the sorting order corresponding to the specific subject information designated before switching, and then determines whether the last one of the sorting orders is completed, if not completed, searches for and designates the subject information corresponding to the sorting order obtained by adding 1 to the sequence number of the sorting order as the specific subject information; if so, exit.

However, it is not necessarily determined whether or not the designation of the last subject information in the sequence is completed, and it may not be determined whether or not the switching instruction operation is received, that is, the subject information corresponding to the sequence number obtained by adding 1 to the sequence number of the sequence is designated as the particular subject information each time the switching instruction operation is received, and if not, the search is continued by adding 1 until the prescribed timing is exceeded or until the CPU8 receives the exit instruction, the information mode is ended.

In this example, returning to step S104, the information designating unit switches the designated specific subject information based on the subject information determined by the task determining unit and the order of the subject information determined by the order determining unit;

Wherein the specified special subject information is switched; in a preferred embodiment, the information designating unit switches the designated individual subject information as the particular subject information from the individual subject information as the particular subject information designated before the switching to the next subject information in the ordered sequence of the subject information as the particular subject information designated before the switching. For example, in this example, the single object information "substation 1 equipment area 1 equipment ic phase" as the specific object information specified before the switching is switched from the single object information "substation 1 equipment area 1 equipment ic phase" as the specific object information specified before the switching to the object information "substation 1 equipment area 1 equipment ic phase" as the specific object information specified before the switching is the next object information "substation 1 equipment area 1 equipment ib phase" in the ordered sequence. Thus, the user can easily perform shooting along the shooting path L11 shown in fig. 11, and the subject is not easily missed.

The display control unit controls the display unit 4 to display the video image and also displays the subject instruction information obtained by the information specification unit switching the specified particular subject information so as to display the particular subject information in a particular display manner. As shown in the display interface 802, the method is switched to display the shot object indication information of 'equipment IB phase of the equipment area 1 equipment I of the transformer substation' specifically; progress information 803 displayed in 801 changes to progress information 804 displayed in 802. Then, the user can take a picture of the corresponding subject based on the subject instruction information of "substation 1 equipment area 1 equipment ib phase" specifically displayed.

The information designating unit also switches the designated individual subject information as the particular subject information from the subject information designated before the switching to the subject information immediately preceding the subject information designated before the switching as the particular subject information in the ordered sequence. The switching is performed based on the sort order, which may be the order of the sort order or the reverse order, and it may be specified whether the switching is performed in the order of the sort order or the reverse order.

Further, when the information specifying unit specifies, as the particular subject information, two or more pieces of subject information adjacent in order; the information specification unit switches the specified object information adjacent to the specified number of specified object information in order from the specified number of specified object information adjacent to the specified number of specified object information before switching to the specified number of specified object information adjacent to the specified number of specified object information next to the specified object information in the sequence of the order.

For example, when "substation 1 equipment section 1 equipment ic", "substation 1 equipment section 1 equipment ib", "substation 1 equipment section 1 equipment ia phase", are simultaneously specified, in response to the switching instruction operation, the switching specification "substation 1 equipment section 1 equipment iia", "substation 1 equipment section 1 equipment iib", "substation 1 equipment section 1 equipment iic phase" is taken as the specific object information.

Further, when the information specification unit specifies the object information, which is the particular object information, as two or more pieces of the object information adjacent in order, the information specification unit may switch the pieces of the object information in the reverse order.

Note that in the present embodiment, the description is given by way of example with a small number of pieces of subject information, and in an actual image capturing operation, the effect of the embodiment of the present invention is remarkable with a large number of subjects.

Similarly, for complex equipment or the situation that multi-angle shooting is needed, related shot object information can be pre-stored and then sequenced for the shot objects (such as components, angles and the like) needing to be shot independently, so that omission of shooting parts can be avoided.

The display unit of the display unit 4 may be configured to have more than one display for displaying an image, or a display for displaying subject information exclusively, and the display control of the simultaneous display by the display control unit may be simultaneous display on the same display or on different displays.

Preferably, the subject indication information is displayed simultaneously with the continuous image; the present invention is not limited to this, and a dynamic image may be displayed on the display unit 4, and when a display instruction of predetermined subject instruction information to be displayed specifically is received, the subject instruction information and the frozen image may be displayed at the same time; further, the object information may be displayed by switching with infrared. In addition, after the special display, blanking may be performed after a predetermined time of display or based on a user operation to avoid occupation of the display screen.

As described above, based on the pre-stored subject information, the sorting order of the subject information is determined based on the sorting factor, then by designating the particular subject information and simultaneously displaying the image and the particular displayed subject indication information on the display section, the user recognizes the subject or views the device signboard with the particular displayed subject indication information as the indication information of the subject currently photographed, reducing the case of false photographing; then, the specific shot object information is switched and designated according to the determined ordering order, so that the shot object indication information is specifically displayed according to the ordering order, the situation of wrong paths or mess is greatly reduced, and the omission is avoided; thereby improving shooting efficiency and reducing working strength. Wherein through the display of statistics shooting progress information, the user can rationally arrange the shooting progress. In summary, example 1 is a preferred embodiment, and of course, it is not necessary for any product embodying an embodiment of the present invention to achieve all of the advantages set forth above at the same time.

Example 2

In the present embodiment, in the image processing apparatus 100 having the same structure as that shown in fig. 1, in the hard disk 5, a control program for setting a filter condition by a user and a control program for determining subject information according to the filter condition, and a control program for executing recording processing are stored; the shooting task at this time is to shoot an object with the type of the object being equipment I in the transformer substation 1; in the past, because the purposes of detection are different, the detection of all the photographed objects of the whole station is available, and the detection of the photographed objects of specific types is available, and the photographed objects of specific types are distributed in different equipment areas, so that a user is required to search for the photographed objects, and the detection is very inconvenient. If the subject information is used as the filtering condition, the user can shoot conveniently.

The present embodiment is described with reference to fig. 12 to 17.

In step S201, the CPU8 performs control to continuously monitor whether or not the user has selected the information mode through the operation unit 7, and when detecting that the user has selected the information mode through the mode button or menu, the routine proceeds to step S202.

Step S202, setting filtering conditions;

The CPU8 controls to cause the display section 4 to display the setting interface shown in fig. 3; when the user selects the filter condition setting item 303 therefrom, and then, the display section 4 displays a filter condition setting interface as illustrated in fig. 13, the user performs setting and determination of the filter condition; then, when the determination of the determination item 300 is detected, the process advances to step S203.

A task setting section for setting a filtering condition (of subject information included in a photographing task) by a user, the filtering condition being composed of a keyword and a filtering logical relationship related to the filtering condition.

The task setting part is provided with a keyword determining unit, a logic determining unit and a filtering condition generating unit, wherein the keyword determining unit is used for determining keywords related to the filtering condition; a logic determination unit for determining a filtering logic relationship of the keywords related to the filtering condition; filtering the logical relationship refers to filtering the non-relationship of a single keyword, and can also be the combination of the and, non-logical relationship and filtering non-relationship of a plurality of keywords; and the filtering condition generating unit is used for generating the filtering condition according to the keywords related to the filtering condition and the specified filtering logic relation.

The keyword related to the filtering condition may be a keyword; a plurality of keywords may also be used, wherein the plurality of keywords includes a representation of a range of keywords, such as a number range, a letter range, a ranking order range, a time range, and the like. In addition, when the subject information has various pieces of information on the subject, such as a attribution unit, a classification level (such as a voltage level, an importance level, etc.), a model, a manufacturer, performance, and characteristics, a history of past photographing or inspection, a manufacturing date, a lifetime, and the like, the keyword may be determined therefrom to set the filtering condition, but these pieces of information are not necessarily used to generate the subject instruction information particularly displayed.

A keyword determination unit configured to determine keywords related to the filtering condition; for example, one or more of the following embodiments may be used to determine keywords related to filtering conditions.

One embodiment, for example, pre-stores keywords for selection, and determines keywords related to the filtering conditions based on the user's selections.

One embodiment, for example, defaults to one or both of the keywords associated with the filter term.

One embodiment, for example, is to enter a keyword by a user, and the keyword entered by the user is determined to be a keyword related to the filtering condition.

One embodiment, for example, provides a selection item of a keyword attribute to be selected, and determines, based on the keyword attribute selected by the user, a keyword conforming to the keyword attribute as a keyword related to a filtering condition; this is the case when some of the subject information has attribute information of a specific attribute, and other parts of the subject information do not have attribute information of the specific attribute, the subject information can be filtered easily.

In a preferred embodiment, a keyword for obtaining attribute information of a predetermined attribute in the subject information is searched for based on the subject information stored in the information storage unit, and a keyword related to the filtering condition is determined based on the selection of the user.

Specifically, the CPU8 controls to cause the display section 4 to display a keyword menu item of a keyword query configured as attribute information representing a prescribed attribute, such as the filtering condition setting interface, the substation 1303, the device area 1304, the device type 1305, and the like as illustrated in fig. 13. When the object information shown in fig. 15 is stored in the information storage section, object information of some other substations is also included, unlike fig. 5. The CPU8 refers to the subject information stored in the information storage unit based on the device type 1305 selected by the user from the current shot "device i", obtains the keywords "device i, device ii, device iii …" of the attribute information of the predetermined attribute (device type), causes the display list 1306, and the user selects "device i" therein, and the keyword determination unit determines that "device i" is the keyword related to the filtering condition. Here, the display section 4, the operation section 7, and the CPU8 constitute an example of a task setting section; in this embodiment, when the number of types of keywords of the attribute information of the subject information is large, the operation is troublesome.

In still another preferred embodiment, the keyword determination unit determines a keyword query condition, and queries, based on the subject information stored in the information storage section, a keyword that obtains attribute information of a prescribed attribute in the subject information that meets the keyword query condition, and determines a keyword related to the filtering condition based on a user's selection; wherein the keyword query condition is one or more of the keywords previously determined to be relevant to the filtering condition.

Specifically, the CPU8 causes the display section 4 to display a filtering condition setting interface in the setting operation as in the example of fig. 14, and after the user selects the substation 1403 through the operation section 7, the CPU8 inquires of the key word of the attribute information of the substation attribute based on the object information stored in the information storage section, causes the display list 1406; from which the user selects "substation 1", the CPU8 determines "substation 1" as a keyword related to the filtering condition; then, when the device type 1405 is selected, the cpu8 determines "substation 1" selected by the user as a keyword query condition, and queries the keywords "device i, device ii, device iii" of the attribute information of the device type attribute conforming to the subject information of "substation 1", so that the list 1407 is displayed, from which the user selects the device type "device i"; the CPU8 determines "substation 1", "equipment i" as keywords related to the filtering conditions. Obviously, this way, the number of keywords to be selected will be simplified.

A logic determination unit for determining a filtering logic relationship of the keywords related to the filtering condition; the filtering logic relation refers to filtering non-relation when the keywords related to the filtering condition are single keywords, or can be a combination of the other or non-logic relation and filtering non-relation when the keywords related to the filtering condition are a plurality of keywords; may be default or may be determined based on user settings.

An example of a single keyword, such as a keyword related to a filtering condition is a single keyword "device i", see fig. 13, and "yes" 1301 and "no" 1302 options of a photographing condition for setting a filtering non-relationship; when 'yes' is selected, combining the filtering condition generated by the keyword 'equipment I' for subsequent searching and determining the object information conforming to the 'equipment I' as a shooting task; when "not" is selected, the filter conditions generated in conjunction with the keyword "device i" are used for subsequent searches to determine subject information conforming to the non-device region 1 ". In this example, the logic determination unit determines the filter non-relationship as "yes" according to the default filter non-relationship as "yes". When the user determines by the operation unit 7, the filtering condition generating means (CPU 8) generates a filtering condition by combining the default filtering non-relation "yes" with the keyword "device i" selected by the user. In the task determination, object information conforming to the keyword "device i" is searched for as determined object information.

For example, as for a plurality of keywords such as "substation 1", "device i", see fig. 14, the logic determination unit determines the filter non-relationship as "yes" according to the default filter non-relationship as "yes", and the logic determination unit determines the logic relationship of the keywords "substation 1", "device i" as "and" according to the default logic relationship (keywords of different attributes are "and" relationship) "and" according to the default logic relationship (keywords of different attributes are "and" relationship "); when the user makes a determination through the operation unit 7, the filtering condition generating means (CPU 8) generates a filtering condition by combining the keywords "substation 1" and "equipment i" related to the filtering condition with a default logical relationship "and with a default filtering non-relationship" yes ". At the time of task determination, object information conforming to the keyword "substation 1" and conforming to "equipment i" is searched as determined object information. However, the logic determination unit is not limited thereto, and the user may perform a logic relationship (not shown) between the setting keywords.

When the setting of the filter conditions is completed, the user determines by the determination key in the operation section, and the generated filter conditions are recorded in the temporary storage section 6 or in the hard disk 5 or the like as a later use. Step S203 is entered.

In step S203, the task determination unit determines the object information from the object information stored in the information storage unit.

If the information storage unit stores the object information shown in fig. 15, for example, the "substation 1" and the "equipment i" are used as the filtering conditions, and the object information matching the keyword "substation 1" and matching the "equipment i" is searched for as the determined object information (the object information shown in fig. 16).

Step S204, a sorting order of the object information is determined.

In the present embodiment, the CPU8 controls to display subject instruction information obtained by a predetermined number of subject information at a predetermined position of the display section 4 in accordance with a default sort factor, as shown at 1701 in fig. 17; the user can sort and adjust the displayed object indication information according to the requirement, and drag the display object indication information through a cross key or a touch screen to arrange the sequence; in addition, information representing the ordering order, such as numbers, may also be entered; for example, the subject instruction information and the sort order shown in 1702 shown in fig. 17 are obtained. Here, the order determination unit determines the order of the object information corresponding to the object instruction information in response to a predetermined operation to adjust the order of the object instruction information displayed on the display unit.

Step S205 of designating special subject information;

After the order adjustment is completed, the user selects the object instruction information "substation 1 equipment area 1 equipment ic phase" shown in 1702 in fig. 17, and the CPU8 designates the object information corresponding to the object instruction information as the special object information, and proceeds to step S206.

Step S206, transmitting the image data received through the communication interface 1 to the temporary storage unit 6; the CPU8 performs predetermined processing on the video data to obtain image data of the video, and stores the image data in the temporary storage unit 6.

In step S207, the object instruction information obtained by the special object information and the video are simultaneously displayed on the display unit 4. Thus, when the user finishes photographing the device I in the device area 1, the user will go to photographing the device I in the device area 3, and the search for the device I in the device area 2 is avoided. And, redundant information of other substations is not displayed either.

In step S208, the CPU8 determines whether or not there is a prescribed instruction operation or a prescribed recording switching condition is satisfied. If not, the steps S206 to 208 are repeated, and the display unit 4 displays the continuous video and the subject instruction information. If so, the flow advances to step S209.

Here, the operation unit 7 has a key configured to represent recording and switching, and realizes an operation capable of performing recording and switching by one-touch (e.g., one action) operation, and the user does not need to select or search for the subject information shown in the display interface in fig. 17, so that the operation is simple.

Further, the CPU8 may be configured to perform the following steps based on other predetermined conditions; for example, when it is determined that the predetermined time interval is satisfied, for example, when the temperature value for the image analysis is in accordance with the criterion threshold (for example, when the temperature value for the image (for infrared thermal image) exceeds the predetermined threshold), or when a predetermined condition such as a trigger signal of another sensor connected to the image processing apparatus 100 is detected, the process proceeds automatically to step S209 even if the predetermined instruction operation from the operation unit 7 is not performed. The predetermined condition includes that even if the predetermined instruction operation is performed, the recording process is performed when other predetermined conditions are satisfied at the same time, for example, a predetermined time interval is satisfied. Obviously, the prescribed condition may be various conditions configured in advance.

Step S209, recording processing.

The CPU8 as a recording unit performs association recording of predetermined video data and predetermined recording information in response to the instruction. And then proceeds to step S210.

A recording unit for recording predetermined video data in association with predetermined recording information, the predetermined video data including, for example,

The predetermined video data is, for example, video data (frames) received by the communication interface 1 in response to a predetermined (recording) instruction operation or a timing when it is judged that a predetermined (recording) condition is satisfied (or a predetermined timing thereafter); for example; in response to a predetermined (recording) instruction operation or a determination of a time (or a predetermined time after) that meets a predetermined recording condition, predetermined image data (e.g., a predetermined one frame) out of the plurality of frames of image data temporarily stored in the temporary storage unit 6; for example, the image data in the above case is obtained after predetermined processing (the predetermined processing is one or more of decompression, correction, interpolation, white balance compensation processing, Y compensation processing, YC conversion processing, pseudo color, pixel reduction, compression, and the like depending on the type of the image capturing apparatus 101 and the image data format); for example, a predetermined number of multi-frame image data are recorded; for example, data (frames) obtained by subjecting a predetermined number of pieces of multi-frame image data to a predetermined process, such as integrating the multi-frame image data stored in the temporary storage section 6, to obtain image data of one frame of image after the process; for example, one or more of the image data obtained in these cases may be used.

Specifically, in one embodiment, in response to a predetermined instruction operation, the CPU8 creates a video file and records the video file on the hard disk 5 in association with information on specific subject information specified in the temporary storage 6, with respect to video data specified in the temporary storage 6.

Specifying recording information, for example, information on specified special subject information, such as specified information in the specified special subject information; the predetermined record information may be all or part of the object information as the specific object information, and may store, for example, "substation 1 equipment area 1 equipment ic phase", or "equipment area 1 equipment ic phase", or may store, for example, ordering order information added to the specific object information.

In the embodiment of the association record, information about the specified specific subject information is added as information of the image data in the predetermined format, see an implementation diagram of the image file structure in fig. 22; the video data 2201 is one frame of video data specified in the video data stored in the temporary storage unit 6 in response to the timing of the specified instruction operation; information (predetermined record information 2202) concerning the specific object information, which is object information representing "substation 1 equipment area 1 equipment ic phase"; other additional information 2203 such as time of shooting, shooting parameters such as ring temperature, distance, etc.

In addition, the association recording process means that information on the specific subject information can be recorded in an information file or an index file associated with the video file, etc., and the CPU8 can generate the information file or the index file; further, the recording unit may further include a file name generating unit configured to generate a file name of the video file, the file name including at least information related to the specified specific subject information; for example, the name of the generated image file: substation 1 equipment area 1 equipment IC phase jpg; preferably, a file name including predetermined classification information is generated from classification information in the subject information, for example, from predetermined attribute information: the method comprises the steps of 1-equipment area 1-equipment I-C phase jpg of the transformer substation, wherein attribute information can be distinguished by adopting a 'separator' mode and the like; further, a file name is generated in combination with the time information, such as substation 1-equipment area 1-equipment I-C phase-20120409. Jpg; the essence of the associated record is that the record is convenient for the subsequent batch analysis, the file name contains the shot object information, so that the user can conveniently check the record, and further, the identifiable file name containing the specified classified information is generated according to the classified information in the shot object information, so that the subsequent batch analysis is convenient for reading and identifying the classified information in the file name.

In step S210, the CPU8 determines whether the task is completed.

The CPU8 judges whether or not the specification of the last subject information in the sorting order is completed; if not, the process returns to step S205, and at this time, the specific subject information designated before the switching is switched to the "substation 1 equipment area 1 equipment ib phase" in accordance with the sort order. If the information mode is finished, words such as 'task completion' can be displayed on the screen, and then the information mode returns to the standby state of the infrared recording device to wait for the operation of entering other modes by a user. Further, the imaging can be performed by switching to a predetermined order start point, for example, a start point of the order.

In this embodiment, task setting is performed after entering an information mode; however, it is not limited thereto, and a task setting may be performed first, for example, a new data file is generated; then entering an information mode; in addition, the task setting may be performed multiple times, and different combinations may be performed to obtain more embodiments.

In the present embodiment, there is a key configured to represent a recording and switching operation, and in response to a prescribed instruction operation of the key, that is, recording processing and switching processing are performed, making the operation simpler; in the present embodiment, the recording process is performed first and then the switching specification is performed, but the switching specification may be performed first and then the specific subject information before the switching specification may be recorded in association with the predetermined image data.

In addition, in the present embodiment, there is a single key configured to represent the record instruction operation and the switch instruction operation, and a corresponding control program, and the CPU8 continuously performs the record processing and the switch processing in response to a prescribed instruction operation (like an operation action) or a prescribed condition of the key; making the operation simpler. In the present embodiment, the association recording process of the object information "substation 1 equipment area 1 equipment ic phase" specified before the switching and the image data is performed first, and then the switching process (switching to the "substation 1 equipment area 1 equipment ib phase") is performed, but this may also be included in the case where the switching specification is performed first, and then the specific object information "substation 1 equipment area 1 equipment ic phase" specified before the switching and the specific infrared data are recorded in association, and the recording section is configured to perform the association recording of the specific infrared data and the information related to the specific object information in response to the recording instruction operation or in accordance with the specific recording condition.

In order to correctly describe the internal working principle and steps, a step-by-step description manner is adopted in the embodiment, but in the practical implementation process, different embodiments can be adopted to optimize the operation for the user operation, for example, the operation is more convenient by continuously performing the recording and switching actions based on the same (secondary) operation and/or according to the specified conditions, and the same operation, for example, the same operation action on the same key, may complete a plurality of steps (such as completing the recording indication operation and the switching indication operation) described above; for example, different operation actions (such as half-pressing and full-pressing) in the same operation of the same key implement one or more steps described above (such as completing the record indication operation and the switch indication operation respectively, or completing the record indication operation and the switch indication operation simultaneously); for example, the operation actions of different keys within a prescribed time (for example, the record indication operation and the switch indication operation may be completed separately, or the record indication operation and the switch indication operation may be completed simultaneously). Further embodiments are available by performing configuration combinations (e.g., function combinations of keys, function combinations of operation actions, etc.) of different switch instruction operations, record instruction operations, etc. The operation is not limited to the key, and may be performed by other operation means such as a touch panel.

As described above, by setting the filtering condition of the subject information in the shooting task, the display of redundant information can be reduced, the number of subject instruction information is made smaller, and the displayed subject instruction information is easy to adjust the sorting order; in addition, the imaging path is clarified in advance, and the purposes of reducing intensity, improving efficiency and avoiding omission of an object are achieved in the subsequent imaging.

Example 3

In the present embodiment, in the image processing apparatus 100 having the same configuration as that shown in fig. 1, another recorded control program and another display control program are stored in the hard disk 5; the current shooting task is to shoot the subject as shown in fig. 19. The present embodiment is described with reference to fig. 18 to 22.

The exemplary list shown in fig. 18 represents an implementation diagram of storing subject information and associated reference image composition data for obtaining image data of a reference image to be displayed together with a video for providing a user with reference to photographing of a subject.

According to the use of the reference image in image capturing, a reference image such as an image showing a predetermined morphological feature of the subject is exemplified, and the subject's physical form represented by the reference image is provided as a visual reference for capturing. It may be various types of subject images such as a visible light image of a subject, a pre-drawn image, or the like, which embody predetermined morphological characteristics. The reference image composition data can be dot matrix data, vector graphic data, data composed of dot matrix data and vector graphic data, and the like. Such as reference images T1 (outline images) and C1 (visible light images) in fig. 18.

Further, a reference image such as an image for illustrating a predetermined analysis region of the subject; which is displayed in a superimposed manner on the image at a predetermined position and a predetermined size. As shown in an analysis pattern F1 composed of three frame-shaped analysis area units in fig. 18, the user is prompted to pay attention to the analysis site corresponding to the subject indicated by the subject indication information "1 substation 1 equipment area 1 equipment ic phase" particularly displayed in the subject information display field 1801. The analysis region configuration data is used to obtain image data of an analysis pattern, and the analysis region configuration data is usually vector pattern data, but may be dot matrix data.

In addition, the reference image can be exemplified by other image information for prompting, such as information for labeling the reference image.

The reference image and the image are displayed together on the same display, for example, the reference image is superimposed on the image at a predetermined transparency, and as shown in fig. 18, the reference image T1 is superimposed on the image; for example, the reference image C1 is combined with the image, as shown in fig. 18, and the reference image C1 is not superimposed on the image, and is displayed in combination with the image for the user to refer to; for example, in one embodiment of the fusion, the reference image and the image are fused (not shown), the transparency of the overlapping of the image data of the reference image and the image data of the image at each pixel is determined based on the color value of the pixel in the image, and the image data obtained after the overlapping is used as the image data of the pixel, thereby obtaining the effect of the fusion of the reference image and the image.

In one embodiment, a reference image is superimposed on a predetermined position in an image at a predetermined size and a predetermined transparency, and a contour image T1 is superimposed on an image IR0, and has a predetermined position and a predetermined size, and indicates a morphological feature of an object image and a position and a size of the object image in the image IR0, so as to prompt a user to pay attention to view a photographing position and an imaging position and a size of an object indicated by object indication information "1 substation 1 equipment area 1 equipment ic phase" particularly displayed in the object information display field 1801.

T1 to T6 illustrated in fig. 19 represent associated reference image configuration data, and different pieces of subject information are associated with different pieces of reference image configuration data; but is not limited thereto, different subject information may be associated with the same reference image composition data. In the related embodiment, for example, the subject information to which the sorting order information is added is set as the file name of the reference image data file; for example, it is preferable that the identification information such as a file name and a number of the reference image configuration data is associated with the subject information, and the information storage unit stores the file of the reference image configuration data corresponding to the identification information, thereby avoiding redundancy of data.

In a preferred embodiment, each piece of subject information is associated with constituent data of a plurality of types of reference images, so that the switching and the combination use are facilitated. And the relative positional relationship (not shown) of the images obtained for each type of reference image composition data associated with the same subject information is saved.

Referring to fig. 20, in the present embodiment, the control steps of the information mode are as follows:

In step S301, the CPU8 continuously detects whether or not the user selects the information mode through the operation unit 7, and if so, the flow proceeds to step S302.

Step S302 and step S303 are similar to the previous embodiments 1 or 2 and are not repeated.

Step S304 designates object information as special object information. The first subject information "substation 1 equipment area 1 equipment ic phase" in the sorting order is designated as the special subject information, and the flow advances to step S305.

In step S305, the video data received through the communication interface 1 is transferred to the temporary storage unit 6, and the cpu8 performs predetermined processing to obtain the image data of the video, and stores the image data in the temporary storage unit 6.

In step S306, the CPU8 controls to display the subject instruction information obtained from the particular subject information specified by the information specification section in a particular display manner while controlling the display section 4 to display the image generated from the image data, and to display the reference image obtained from the reference image configuration data associated with the particular subject information.

In the present embodiment, the video, the subject instruction information obtained by the specified special subject information, and the reference image obtained from the reference image configuration data associated with the special subject information are displayed together. As shown in a display interface 2101 in fig. 21, the CPU8 as a display control section controls to display, in order, object instruction information obtained by a prescribed number of object information, underline marks of "substation 1 equipment section 1 equipment ic phase" so as to be distinguished from other object instruction information, and also simultaneously displays a reference image T1 obtained by the object information "substation 1 equipment section 1 equipment ic phase" associated reference image configuration data.

Specifically, in one embodiment of display control, the configuration data of the reference image T1 associated with the device area 1 device ic phase of the substation 1 device area 1 is read from the hard disk 5 based on the specified object information, and the image data of the obtained video is synthesized based on predetermined display configuration parameters (for example, parameters such as the content of the image data to be displayed on the display interface, the position, color, size, transparency, and superimposition order of the display of each content), the image data of the specified specific object information to obtain the image data of the object instruction information, and the image data of the reference image T1 obtained from the reference image configuration data associated with the specific object information; storing the synthesized image data in the temporary storage unit 6; the control display control unit 3 displays the synthesized image data on the display unit 4. As shown in fig. 21, this composition is to superimpose the reference image on the video.

Wherein, when a plurality of types of reference image composition data are associated based on the object indication information obtained from the object information, display configuration data such as reference image type, transparency, superimposition order, and the like for obtaining composition and display of the object indication information in the object information can be prescribed in advance. The display configuration data may be default, or some or all of the display configuration data may be configured by a user.

The display control unit controls the image data of the reference image obtained from the reference image configuration data associated with the specified specific subject information to be superimposed at a predetermined position in the video image according to a predetermined size. The reference image is located at a predetermined position and a predetermined size of the image, and may be obtained in various embodiments.

For example, the display control unit may be configured to determine a predetermined position and a predetermined size represented by the position information, and the reference image obtained for the reference image configuration data may be positioned at the predetermined position and the predetermined size in the video, and the CPU8 may be configured to combine the reference image obtained based on the predetermined size with the predetermined position in the video.

In addition, the position, size or also the rotation angle of the reference image may be determined according to the user input through the operation section.

Further, the display area is determined by a predetermined adaptive display area. The adaptive display area is an area determined in the image and used for adaptively displaying the reference image, and the adaptive display refers to the specified position of the reference image in the adaptive display area, so that the maximization display with non-overflow and fixed aspect ratio in the adaptive display area is performed. Specifically, the CPU8 (display control section) is configured to perform calculation of a predetermined position and a predetermined size in the image after the adaptive scaling of the reference image, taking as an example that the set adaptive display area is centered in the image and the reference image is displayed "reference image centering" in the adaptive display area, the CPU8 calculates an X-axis and Y-axis ratio of the adaptive display area (sizes X1, Y1) to the reference image (reference image sizes X2, Y2 before scaling), and selects a smaller one of the X1/X2 and Y1/Y2 as a scaling ratio based on a center point of the reference image when the reference image is centered, thereby obtaining the predetermined position and the predetermined size in the image of the adaptively displayed reference image. The effect of the reference image T1 being displayed "reference image centered" in the adaptive display area Z1 is illustrated in fig. 18.

Further, the reference image adaptive display also includes a case where the CPU8 is configured to combine a plurality of constituent data as a combination object and perform calculation of the position and size of the adaptive display of the combined reference image. It is to be noted that the image generated by each type of constituent data participating in the combination is not necessarily displayed.

Further, based on the relative positional relationship between the images obtained by the reference image configuration data, the position and the size in the picture of the image generated by the other configuration data are obtained from the position and the size in the picture of the image obtained by one of the configuration data.

In step S307, the CPU8 determines whether there is a recording instruction operation. If not, the process proceeds to step S309, and if the record instruction operation is given, the process proceeds to step S308.

As shown in the display interface 2101, the user checks and photographs the corresponding subject with the subject instruction information "substation 1 equipment area 1 equipment ic phase" and the reference image T1 (outline image) displayed specifically as the presentation information for photographing the current subject. In the display interface 2101, there is a difference in position and size between the subject image IR1 and the contour image T1; it is conceivable that the form of the subject image IR1 taken by the user (in practice, many users often take only an incomplete subject image, or an inappropriate photographing angle) and the imaging position, size, and angle thereof in the image are difficult to subjectively grasp, if there is no subject prompt information and reference image prompt means. Then, the user visually sets the image forming position and size matching state between the object image IR10 and the reference image T1 in the display interface 2102 of fig. 21, which is obtained after adjustment, by changing the image forming position and adjusting the image forming distance, image forming position and angle between the optical component of the image photographing apparatus 101 and the object "substation 1 equipment area 1 equipment ic phase", based on the reference of the contour image T1; in this case, the user can perform status determination or recording of the subject image IR10 conforming to the predetermined form, and the result of the status evaluation is easily obtained due to the subject image IR10 specification. When the user completes shooting of the corresponding object according to the object indication information "the equipment ic phase of the equipment area 1 of the substation 1", the user then presses a key representing the record of the operation part 7 to perform the record indication operation.

Further, the CPU8 may be configured to perform recording based on other predetermined recording conditions; for example, when it is determined that a predetermined time interval is satisfied, for example, when a temperature value in an image (infrared image) exceeds a predetermined threshold value, for example, when a criterion threshold value is satisfied in image analysis, for example, when a predetermined condition such as a trigger signal of another sensor device connected to the image processing apparatus 100 is detected, the process proceeds to step S308 automatically even if a recording instruction operation from the operation unit 7 is not performed.

Step S308, recording processing.

The CPU8 performs association recording of predetermined video data and predetermined recording information; and then proceeds to step S309.

Unlike embodiment 2, reference image composition data (or identity information of composition data) associated with object information may be recorded in addition to all or part of the information of the object information as special object information; specifying the record information, which may include one or more of the following;

for example, all information of object information, or part of the information thereof, which is the special object information;

For example, information related to reference image composition data. Reference image composition data associated with specific subject information as specified, or identity information of the composition data; wherein when the specified specific subject information associates a plurality of types of reference image composition data, one or more of them may be recorded, wherein the recorded reference image composition data is not necessarily composition data of the obtained displayed reference image; for example, in the present embodiment, the reference image configuration data T1 (the displayed reference image T1 is obtained) may be recorded, and when the analysis region configuration data is also associated with the special subject information, the analysis region configuration data (even though not shown) or the identity information thereof may be recorded. Obviously, the configuration data of other objects (for example, the center line of the contour image T1, the outsourcing rectangle, etc.) having a relative positional relationship with the contour image T1, which are generated by the reference image, for example, the contour image T1, according to a predetermined rule algorithm may be recorded.

For example information about the position parameters of the reference image in the picture. The reference image composition data associated with the specific subject information as specified constitutes position information of an image obtained from the data in the video. In this case, when the specified specific subject information is associated with a plurality of types of reference image configuration data, the positional information of the image obtained from one or more of the configuration data in the video may be recorded, for example, in the present embodiment, the positional parameter of the reference image T1 in the video may be recorded, and when the specific subject information is also associated with the configuration data of the analysis region, the positional parameter of the analysis region in the video (even if not shown) may be recorded. Obviously, the position parameters of other objects (for example, the center line of the outline image T1, the outsourcing rectangle, etc. generated from the outline image T1) which have a relative positional relationship with the outline image T1 and are generated by the reference image, for example, the outline image T1, according to a predetermined rule algorithm may be recorded.

Referring to fig. 22, in the present embodiment, the image data 2201 is predetermined one frame of image data in the temporal storage unit 6 at that time, and corresponds to the image IR0 shown in fig. 21 at 2102, and since the reference image T1 is taken as a reference, the subject image IR10 is located at a predetermined position in the image IR0 and has a predetermined size; the and prescribed record information 2202 includes subject information representing "substation 1 equipment area 1 equipment ic phase", reference image T1 configuration data, and position information of reference image T1 in image IR 0; other additional information 2203 such as the time of shooting, etc.

In step S309, the CPU8 determines whether or not there is a switching instruction operation, and if not, returns to step S305, and if there is a switching instruction operation, proceeds to step S310.

In step S310, the CPU8 determines whether the task is completed. The CPU8 judges whether or not the specification of the last subject information in the sorting order is completed; if not, the process returns to step S304, and the specified special subject information is switched in the order. If so, ending the information mode.

Further, the subject information and the reference image composition data associated therewith do not have to be stored in advance in the same storage medium physically; for example, the identity information of the subject information and the corresponding reference image configuration data are stored in the storage medium of the image capturing apparatus 101, and the reference image configuration data corresponding to the subject information is stored in an external device such as a memory, a computer, or the like connected by wire or wirelessly through the communication I/F8, and when the specific subject information is specified, the CPU8 issues an instruction to the external device to obtain the reference image configuration data corresponding to the subject information by wire or wirelessly.

As described above, by specifying and sorting the subject information corresponding to the photographed subjects based on the pre-stored subject information and the associated reference image configuration data, specifying the particular subject information therein, and particularly displaying the subject instruction information, the reference image, and the video obtained by the particular subject information on the display section 4, the user can recognize the subject or view the device signboard with the particularly displayed subject instruction information as the instruction information of the subject currently photographed, thereby reducing the occurrence of false photographing; when the record instruction operation is performed, namely, the information related to the image data and the special object information is recorded, then the specified special object information is switched according to the ordering order, so that the operation is simple; the reference image which is presented in the image, has a specified position and a specified size and shows the outline characteristics of the shot object, provides visual reference for a user to shoot the shot object image, and prompts and standardizes the shooting angle, the shooting position and the shooting distance of the shot object; obviously, according to the shot indication information and the reference image reference, the shooting requirements of a user are clear at a glance, the shooting difficulty can be greatly reduced without excessively relying on experience accumulation and subjective ideas, the detection quality and speed are improved, a common user can achieve a good shooting skill level, and the information related to the special shot information is recorded. Of course, it is not necessary for any of the products embodying the embodiments of the invention to achieve all of the advantages set forth above at the same time.

Example 4

The difference from the above embodiment is that the hard disk 5 stores therein the reference image configuration data and its associated information as shown in fig. 23; the present embodiment determines the ordering order of the reference image composition data according to the association information of the reference image composition data; and further, the specification and the switching display of the reference image composition data are realized.

Reference pattern will be described with reference to fig. 24:

step S401 of determining the sorting order based on the related information (not limited to the number of the sorting order, such as time, number, etc., or the subject information may be related, but there may be a plurality of reference images related to the same subject information and having different numbers of the sorting order, such as reference images representing different angles of the subject, and subject information being the same or different); the ordering of the reference pictures may be determined, for example, from ordering information associated with the reference picture composition data; or the related information of the reference images is sequenced according to a specified sequencing rule to determine the sequencing order of the reference images, and the reference images can be sequenced according to keywords in the related information, which is similar to the sequencing mode in the embodiment, and the description is omitted; or the reference image composition data (e.g., displayed as thumbnails) may also be manually ordered; or the associated information may be manually ordered.

When task determination is also required, the reference image composition data related to the subsequent switch specification may be determined by filtering based on keywords or the like in the associated information as filtering conditions, or manually selecting the associated information or the reference image composition data.

In step S402, the reference image composition data is specified, and the information specification unit (CPU 8) is configured to automatically specify the reference image composition data based on the composition data stored in the information storage unit (e.g., automatically specify the ranking first digit) based on the ranking order, or the reference image composition data may be specified by user selection, default mode, or the like.

Step S403 of transferring the image data obtained by the photographing section 1 to the temporary storage section 6;

In step S404, the CPU8 controls to display the specified reference image configuration data and the reference image obtained by the specified reference image configuration data, while controlling the display unit 4 to display the video generated by the video data. As shown in fig. 25, after the configuration data (T1) of the first order is specified, a T1 contour image is displayed. In fig. 25, each sort order 2501 is shown, but it is also possible to display no thumbnail or the like corresponding to the reference image configuration data corresponding to the sort order.

Further, the reference image configuration data may further include subject instruction information concerning a subject, and the subject instruction information displayed is as shown in fig. 26, but unlike the above embodiment, the T1 configuration data includes contents of 2601, such as information representing a subject point, a type, a number, or the like (similar to the contents of the subject information described in embodiment 1), and the subject instruction information is, for example, a component of a vector pattern, or a component of a dot matrix image. Thus, a word of the subject indication information, such as a watermark or the like, can be embodied in the reference image; if there is no subject information associated, when task determination is performed in the present embodiment, unlike the filtering of the above-described embodiment, it is inconvenient to set the filtering condition of the subject instruction information, but filtering may be performed according to the associated information of the reference image configuration data, such as the number, time, or the like. The effect in use is similar to the above-described embodiment.

In step S405, the control unit determines whether or not there is a switching instruction operation. If not, repeating steps S403-S405, and displaying the reference image T1 together with the continuously moving image (continuous composition). If a switching instruction operation from the operation unit 7 is detected, the process advances to step S406. Step S406, the CPU8 determines whether the task is completed; if not, the process returns to step S402, and at this time, the switching is performed in the order of the order based on the reference image configuration data stored in the information storage unit.

In the above-described embodiment, the image processing system using the vehicle as the carrier has been described, but the present invention is not limited to this, and the image processing system including the image processing apparatus 100 and the image capturing apparatus 101 connected thereto may be used as a portable apparatus for practical applications such as hand-held capturing. In the above embodiment, the subject application in the power industry is exemplified as a scene, but the present invention is also applicable to various industries of image capturing. In addition, the processing and control functions of some or all of the components in the embodiments of the present invention may also be implemented with dedicated circuits or general-purpose processors or programmable FPGAs. The foregoing description is merely illustrative of specific embodiments of the invention, and the various illustrations are not limiting of the true spirit of the invention, other modifications and variations may be made to the specific embodiments by those skilled in the art without departing from the spirit and scope of the invention.

Claims

1. An image processing apparatus comprising:

a communication unit configured to continuously receive video data;

The information storage unit stores subject information and associated reference image configuration data;

different subject information may be associated with the same or different constituent data of the reference image;

the display control unit is configured to control a reference image obtained from reference image configuration data based on reference image configuration data associated with subject information, to be superimposed at a predetermined position in an image in accordance with a predetermined size and a predetermined transparency;

The reference image respectively reflects the specific morphological characteristics and/or analysis areas of the shot object corresponding to the shot object information; the reference image is used for providing a reference for prompting and standardizing shooting of one of shooting angles, shooting positions and shooting distances of the shot objects corresponding to the shot object information;

the standard shooting is to enable the image of the shot object to be in an imaging position or a size basically matched state with the reference image visually by changing the shooting position or adjusting an image device;

the information designating unit switches the designated specific subject information in the order of the subject information in response to a switching instruction operation or in accordance with a predetermined switching condition.

2. An image processing apparatus comprising:

a communication unit configured to continuously receive video data;

A display control unit configured to control the display unit to display the image generated by the image data continuously, and to display a reference image obtained based on the specified reference image configuration data;

The display control part is used for controlling the reference image obtained by the reference image composition data to be overlapped at a preset position in the image according to a preset size and a preset transparency;

the reference images respectively reflect the specific morphological characteristics and/or analysis areas of the corresponding shot objects;

the reference image is used for providing a reference for prompting and standardizing one of shooting angles, shooting positions and shooting distances of the shot object;

The information designating unit switches the designated reference image configuration data based on the order of the reference image configuration data in response to a switching instruction operation or in accordance with a predetermined switching condition;

the display control unit is configured to control the display unit to display the video generated by the video data continuously, and to display the reference image obtained by switching the specified reference image configuration data according to the information specification unit.

3. The image processing apparatus according to claim 2, comprising

A task specifying unit configured to specify reference image configuration data from the reference image configuration data stored in the information storage unit;

A ranking determining section for determining a ranking order of the reference image configuration data determined by the task determining section according to a predetermined ranking factor, wherein the ranking order is determined by ranking based on the association information of the reference image configuration data;

The information designating section designates, from the reference image configuration data determined by the task determining section, reference image configuration data for obtaining the reference image, and switches the designated reference image configuration data based on the order determined by the order determining step in response to a switch instruction operation or in accordance with a prescribed switch condition.

4. The image processing apparatus according to claim 1, comprising a task determination section for determining object information from among the object information stored in the information storage section;

The information specification section for specifying subject information as particular subject information from among the subject information specified by the task specification section; the information designating unit is configured to switch the designated specific subject information based on the subject information determined by the task determining unit in response to a switching instruction operation or in accordance with a predetermined switching condition, in accordance with the order of the subject information;

The information specifying unit specifies, as subject information, two or more pieces of subject information adjacent in order; the information specification unit switches the specified object information adjacent to the specified number of specified object information in order from the specified number of specified object information adjacent to the specified number of specified object information before switching to the specified number of specified object information adjacent to the specified number of specified object information next to the specified object information in the sequence of the order.

5. The image processing apparatus according to claim 4, wherein the task determination section is configured to determine the subject information that meets the filtering condition based on the subject information stored in the information storage section, based on a predetermined filtering condition.

6. The image processing apparatus according to claim 5, comprising a task setting section for setting a filtering condition of the subject information; the task setting part is provided with a keyword determining unit, a logic determining unit and a filtering condition generating unit; wherein, the keyword confirms the unit, is used for confirming the keyword related to filtering the condition; a logic determination unit for determining a filtering logic relationship of the keywords related to the filtering condition; and the filtering condition generating unit is used for generating the filtering condition according to the keywords related to the filtering condition and the filtering logic relation.

7. The image processing apparatus according to claim 4, wherein the ranking determining section determines the ranking order for the subject information determined by the task determining section based on a predetermined ranking factor.

8. The image processing apparatus according to claim 7, wherein there is a ranking setting section for a user to set a ranking factor of the subject information; the ranking setting section has a keyword determination unit for determining keywords related to ranking; a ranking factor determining unit for determining ranking factors of the keywords; the ranking setting section is configured to combine the first keyword ranking factor or the first keyword ranking factor with the auxiliary keyword ranking factor to obtain a ranking factor of the subject information.

9. The image processing apparatus according to claim 1, wherein,

The recording device comprises a recording part for performing recording processing for performing association recording of prescribed video data and prescribed recording information including at least information on reference image composition data and/or position parameters of a reference image in a video in response to a recording instruction operation or in accordance with prescribed recording conditions; or the predetermined record information contains at least information on the specified specific subject information.

10. The image processing apparatus according to claim 1, comprising a recording unit configured to perform recording processing in response to a recording instruction;

When the specified specific subject information is associated with a plurality of types of reference image composition data, one or more of them may be recorded;

The subject information is associated with the composition data of a plurality of types of reference images, and the images obtained by the composition data have a prescribed positional relationship;

The composition data of the reference image can be selectively displayed.

11. The image processing apparatus according to claim 1 or 2, wherein the communication unit is configured to receive the image data by wireless, the image data being at least thermal image data;

the reference image embodies the contour features of the subject.

12. An image processing system comprises an image processing device and an image shooting device connected with the image processing device, wherein the image shooting device shoots and outputs image data, and the image processing device receives the image data; the image processing apparatus according to any one of claims 1 and 2.

13. An image processing method, comprising:

A communication step for continuously receiving image data;

The display control step is used for controlling the reference image obtained by the reference image composition data to be superimposed at a specified position in the image according to a specified size and a specified transparency on the basis of the reference image composition data associated with the object information;

The reference image respectively reflects the specific morphological characteristics and/or analysis areas of the shot object corresponding to the shot object information;

The reference image is used for providing a reference for prompting and standardizing shooting of one of shooting angles, shooting positions and shooting distances of the shot objects corresponding to the shot object information;

the information designating step of switching the designated special subject information in the order of sorting the subject information based on the subject information stored in the information storage section in response to a switching instruction operation or in accordance with a prescribed switching condition;

the information specifying step specifies subject information as particular subject information when two or more pieces of subject information are adjacent in order; an information specifying step of switching, from the specified number of pieces of subject information specified before switching, that are adjacent to each other in order, to the specified number of pieces of subject information specified before switching, that are adjacent to each other in order, that are next subject information in order, that are adjacent to each other in order, that are next subject information in order, that are specified before switching, that are adjacent to each other in order;

A recording step of performing recording processing in response to the recording instruction; when the specified specific subject information is associated with a plurality of types of reference image composition data, one or more of them may be recorded; the subject information is associated with the composition data of a plurality of types of reference images, and the images obtained by the composition data have a prescribed positional relationship;

The composition data of the reference image can be selectively displayed.

14. An image processing method, comprising:

A communication step for continuously receiving image data;

an information specifying step of specifying reference image composition data for obtaining a reference image based on the reference image composition data stored in the information storage section;

a display control step of controlling a display section to continuously display the video generated by the video data, and displaying a reference image obtained based on the specified reference image configuration data;

the display control step is used for controlling a reference image obtained by composing data of the reference image, and superposing the reference image at a specified position in the image according to a specified size and a specified transparency, wherein the reference image respectively represents a specified morphological feature and/or an analysis area corresponding to the shot object; a reference for providing a prompt for one of a photographing angle, a photographing position, and a photographing distance of a subject and performing standard photographing;

The reference image reflects the outline characteristics of the shot object; the standard shooting is to enable the outline of the shot object image displayed by the adjusted display part and the outline of the reference image to be in an imaging position or a size matching state visually by changing the shooting position or adjusting the image device; the information designating step of switching the designated reference image configuration data based on the order of ordering the reference image configuration data in response to a switching instruction operation or in accordance with a prescribed switching condition; the display control step is to control the display unit to display the image generated by the image data and to display the reference image obtained by switching the specified reference image configuration data according to the information specification step.