JP7395137B2 - Head-mounted temperature distribution recognition device - Google Patents

Description

The present invention relates to a head-mounted temperature distribution recognition device, and particularly to a head-mounted temperature distribution recognition device using mixed reality (MR) technology.

Generally, in fires in buildings and structures, the percentage of fires caused by electrical equipment (wiring equipment, electrical equipment, lights, telephones, etc., and their electrical parts, etc.) accounts for just under 20% of the total. Furthermore, even if the building does not cause a fire, power cables and the like are often scorched, producing smoke and strange odors, and causing a commotion. In order to prevent fires caused by such electrical equipment, it is necessary to detect overheating abnormalities in the electrical equipment at an early stage.

In order to detect overheating abnormalities in electrical equipment, for example, periodic inspections once a year and daily patrol inspections are carried out. Typically, during periodic inspections, the supply of electricity to the equipment is stopped and the withstand voltage and insulation resistance are measured. Also, during routine inspections, current and voltage are measured while electrical equipment is in operation. Furthermore, in such inspections, a thermo gun or a thermo camera is used to visualize the temperature distribution of the suspected heat generation area as a thermal image (thermography).

Incidentally, in recent years, various devices using a technology called mixed reality have been proposed. Mixed reality is a technology that mixes real space and virtual space to create a space where real and virtual things merge in real time. Patent Document 1 below discloses a head mounted display (integrated head mounted eyepiece) using mixed reality technology.

Further, Patent Document 2 listed below discloses a head-mounted smart device (HMSD) configured so that a user can view a display that displays a thermal image produced by a thermal image forming camera with one eye of the user.

US Patent Application Publication 2019/0025587 Special Publication No. 2017-519999

While technological improvements and widespread use of electrical equipment have made people's lives more convenient and enriching, we now face situations in which lives and property are lost due to electrical fires, and important equipment and valuable cultural heritage are destroyed. Since the losses caused by fire are large, it is necessary to eliminate electrical fires, but as mentioned above, electrical fires account for a large proportion of all fires.

One of the causes of fires specific to electrical equipment is poor connection of power cables, etc. For example, when on-site workers connect three-phase, three-wire power cables one by one manually, there are rare cases where construction defects occur due to insufficient crimping of power terminals or insufficient tightening of screws. These poorly installed areas may have high resistance, and overheating may occur at unpredictable timing after energization. That is, the temperature of the power supply connection portion increases over time and exceeds the temperature limit of the cable sheathing, causing smoke and ignition, which spreads to surrounding combustible materials and spreads the fire.

Therefore, inspection of electrical equipment is very important, but the regular inspection method of stopping the supply of electricity to the equipment and measuring the withstand voltage and insulation resistance does not involve physically touching the electrical equipment directly. While poor connections and loose screws can be found and corrected, abnormalities that generate heat when electricity is turned on cannot be easily discovered because the temperature returns to room temperature after a power outage.

Furthermore, looking at cases of electrical fires in the past, there have been cases in which heat generation and smoke emitted immediately after electricity was turned on after construction work was completed, and cases in which there was an incubation period that led to overheating several months or even years later. Periodic inspections once a year would not be able to discover all of these latent overheating points in advance, and many electrical fire accidents could have been prevented by routine equipment inspections. I understand.

However, the routine inspection method of measuring current and voltage while electrical equipment is in operation requires inspectors to work close to the electrical components inside the equipment, so they are constantly exposed to the risk of electric shock, etc. They are next to each other. In fact, due to misconceptions or carelessness on the part of workers, they accidentally touch live electrical equipment and get electrocuted, resulting in serious accidents resulting in burns and death.

In addition, with the method of using a thermo gun or thermo camera to visualize the temperature distribution of a suspected fever area as a thermal image (thermography) and inspecting for abnormal areas, the inspector must approach the electrical components in the electrical equipment. This is not necessary and the location of the heat generation can be safely identified, but in actual work, the inspector has to hold a thermo gun in his/her hand and aim at the location where the heat is suspected while searching around the area. Even the slightest abnormality lurking inside was overlooked.

In addition, inspectors must hold a thermo gun in their hands and walk around the site with steps, use a stepladder, etc. to get to a high place, and open doors until they can measure the temperature of a hot spot. Hand-held thermo guns and the like pose a safety issue, as your hands are occupied, which can lead to serious injuries if you fall.

Furthermore, although thermo guns and thermographs used in thermal imaging diagnosis are currently commercially available and easily available, specialized knowledge and experience are required to learn techniques for measuring and inspecting electrical equipment. It wasn't always easy to use. Another problem inherent to thermal images is that it is difficult to understand which part of reality the displayed thermal image is showing.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to propose a device for accurately, safely, and/or easily detecting abnormal heat generation in electrical equipment that may cause a fire.

More specifically, one of the objects of the present invention is to accurately, safely, and/or easily detect abnormal heat generation in electrical equipment without requiring an inspector to approach the electrical equipment that may cause a fire. An object of the present invention is to provide a head-mounted temperature distribution recognition device.

The present invention for solving the above-mentioned problems includes the following invention specific matters and technical features.

The present invention according to a certain aspect includes a head mount unit that can be attached to a user's head, and a heat source that is arranged to substantially match the direction of the line of sight of the user wearing the head mount unit, and that generates heat from at least a heat source in a target space. a sensor including a thermosensor for detecting infrared rays and a spatial recognition sensor for recognizing a three-dimensional shape in the target space; a display disposed in front of the user's line of sight; the sensor and the display; This is a head-mounted temperature distribution recognition device that includes a controller that controls the temperature distribution. In the head-mounted temperature distribution recognition device, the controller performs thermography in the target space based on first data output from the thermosensor and second data output from the space recognition sensor. Generate image data that includes Further, the controller performs control so that an image based on the generated image data is displayed on the display.

The controller also includes a temperature data processing unit that generates temperature distribution data based on first data output from the thermosensor, and a temperature data processing unit that generates temperature distribution data based on second data output from the spatial recognition sensor. a spatial orientation estimation unit that generates orientation data; and an image data generation unit that generates the image data including the thermography so that the thermography based on the temperature distribution data matches an object in the target space based on the spatial orientation data. and a section.

Further, the controller may be configured to be able to set a display temperature range in the thermography. The image data generation unit may generate the image data including the thermography according to the set display temperature range.

Further, the controller may be configured to notify the user when the temperature indicated by the first data output from the thermosensor is equal to or higher than a predetermined warning temperature.

Further, the image data generation unit may generate the image data such that a temperature lower than the predetermined warning temperature becomes invisible in the thermography.

Further, the thermosensor may be detachably attached to the head mount section. Further, the sensor may include a pair of thermosensors arranged to correspond to binocular parallax of the user so that the user perceives three-dimensional thermography.

In addition, in thermography for a plurality of electrical components of the same type (for example, a three-phase power supply connection), the controller is configured to set a high temperature when determining that there is a predetermined temperature difference between the plurality of electrical components. The electrical component shown can be determined to have abnormal heat generation. Further, the controller may be configured to warn the user when determining that there is an abnormality in heat generation.

Note that in the present disclosure, "means" does not simply mean physical means, but also includes cases in which the functions of the means are realized by software. Further, the function of one means may be realized by two or more physical means, or the functions of two or more means may be realized by one physical means.

In addition, in this disclosure, a "system" includes a logical collection of multiple devices (or functional modules that realize a specific function), and each device or functional module is physically treated as a single entity. It does not matter whether it is constituted or constituted as a separate object.

According to the present invention, an inspector can quickly, accurately, safely, and/or easily discover abnormal heat generation in electrical equipment that may cause a fire.

Other technical features, objects, effects, and advantages of the present invention will be made clear by the following embodiments described with reference to the accompanying drawings.

1 is a diagram showing an example of the external configuration of a head-mounted temperature distribution recognition device according to an embodiment of the present invention. 1 is a block diagram showing an example of a functional configuration of a head-mounted temperature distribution recognition device according to an embodiment of the present invention. FIG. 2 is a diagram for explaining an example of an operation mode of a head-mounted temperature distribution recognition device according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of a thermography displayed on a display of a head-mounted temperature distribution recognition device according to an embodiment of the present invention. It is a figure showing an example of the menu window displayed on the display of the head-mounted temperature distribution recognition device concerning one embodiment of the present invention. It is a figure showing an example of the display setting screen displayed on the display of the head-mounted temperature distribution recognition device concerning one embodiment of the present invention. It is a figure showing an example of the temperature setting screen displayed on the display of the head-mounted temperature distribution recognition device concerning one embodiment of the present invention. It is a figure showing an example of the mode setting screen displayed on the display of the head-mounted temperature distribution recognition device concerning one embodiment of the present invention. FIG. 3 is a diagram showing an example of thermography with/without a temperature mask displayed on the display of the head-mounted temperature distribution recognition device according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of thermography during image recording displayed on the display of the head-mounted temperature distribution recognition device according to an embodiment of the present invention. FIG. 2 is a diagram for explaining an example of use of a head-mounted temperature distribution recognition device according to an embodiment of the present invention. FIG. 2 is a diagram for explaining an example of use of a head-mounted temperature distribution recognition device according to an embodiment of the present invention.

Embodiments of the present invention will be described below with reference to the drawings. However, the embodiments described below are merely examples, and there is no intention to exclude the application of various modifications and techniques not specified below. The present invention can be implemented in various ways (for example, by combining the embodiments) without departing from the spirit thereof. In addition, in the description of the drawings below, the same or similar parts are denoted by the same or similar symbols. The drawings are schematic and do not necessarily correspond to actual dimensions or proportions. The drawings may also include portions that differ in dimensional relationships and ratios.

FIG. 1 is a diagram showing an example of the external configuration of a head-mounted temperature distribution recognition device according to an embodiment of the present invention. As shown in the figure, the head-mounted temperature distribution recognition device 1 of this embodiment is, for example, a head-mounted wearable device that can be mounted on a person's head. The head-mounted temperature distribution recognition device 1 includes, for example, a head-mounted section 10, a controller 20 (see FIG. 2) that performs various computing processes, a glasses-shaped display 30 having a display function, and various sensors 40. Equipped with The head-mounted temperature distribution recognition device 1 is capable of providing an interactive mixed reality (MR) space by displaying thermography imaged by the sensor 40a and image-processed by the controller 20 on the display 30 so that the user can view it. It is composed of

The head mount unit 10 is configured so that the head mount type temperature distribution recognition device 1 can be stably mounted on the user's head. In this example, the head mount section 10 has a helmet shape assuming use at a work site, but is not limited to this, and may be configured, for example, by a headband or a frame that surrounds the head in the circumferential direction. .

The controller 20 is a computing device that centrally controls the operation of the head-mounted temperature distribution recognition device 1 . As is obvious to those skilled in the art, the controller 20 includes a chip module including a processor, memory, and the like. The controller 20 may be physically configured as a single unit or may be configured as several separate units. For example, the controller 20 may include a first controller that processes temperature distribution data, which will be described later, and a second controller that performs processing to provide mixed reality to the user. In this example, the controller 20 is provided behind the head mount section 10, but it may be provided so as to be housed inside the head mount section 10.

The display 30 is, for example, a glasses-type or goggle-type transmissive display that displays an image that can be viewed by a user wearing the head mount unit 10 . Through the display 30, the user can visually recognize the situation in the real space (visual space) in the viewing direction, and under the control of the controller 20, can visually recognize the images displayed on the display 30 in a superimposed manner.

The sensor 40 is configured to include a plurality of sensor elements that operate under the control of the controller 20 to detect various physical quantities such as infrared rays and visible light. Sensor 40 may be a passive sensor or an active sensor. In this example, the sensor 40 includes, for example, a thermosensor 40a that detects infrared rays from a heat source, a visible light sensor (image sensor) 40b that detects visible light, and/or a depth sensor (not shown). However, it may include a three-dimensional acceleration sensor, a geomagnetic sensor, a GPS sensor, and the like. Sensor 40 may also include an active infrared sensor (not shown) that emits infrared light. The thermosensor 40a is removably provided using, for example, a snap fit. Thereby, by attaching the detached thermosensor 40a to the tip of the long shaft, the user can check the state of temperature distribution in a blind spot area such as an attic space, for example. The sensor 40 constitutes, for example, an eye tracker, an image recognition sensor, a space recognition sensor, a distance measurement sensor (depth sensor) using TOF (Time of Flight), etc., either singly or in an appropriate combination (see FIG. 2).

For example, the sensors 40 may be provided in a pair configuration to enable stereoscopic display using human binocular parallax. That is, the thermal images captured by the two thermosensors 40a placed apart by the distance between the eyes are slightly different images for the right eye and the left eye. Therefore, when viewing this as an MR image, the brain perceives the binocular parallax of the object as a three-dimensional image, giving a more realistic feel to the scene, making it easier to identify fever points, and improving the effectiveness of inspection patrols. Become.

As an example, a plurality of thermosensors 40a are provided in pairs in front of the head mount section 10. As another example, a plurality of thermosensors 40a may be provided on the head mount section 10 in order to detect the temperature of heat sources not only in front of the user but also in the left and right and/or upward directions. As a result, blind spots in the temperature detection direction can be reduced, and abnormal heat generation in electrical equipment can be effectively monitored. Therefore, conventional thermo guns etc. can only measure with a narrow viewing angle in a single direction, but by operating multiple thermo sensors 40a at the same time as in this example, the detection area can be expanded and abnormal locations can be overlooked. It will be possible to prevent this.

Although not clearly shown in the figure, the head-mounted temperature distribution recognition device 1 may include a speaker and a vibrator that function as a user interface (see FIG. 2). A speaker or a vibrator is attached to the head mount section 10, for example. Under the control of the controller 20, the speaker outputs a predetermined warning sound or voice guidance, and the vibrator vibrates.

FIG. 2 is a block diagram showing an example of the functional configuration of the head-mounted temperature distribution recognition device 1 according to an embodiment of the present invention. In the figure, components particularly relevant to the present disclosure are shown among various components of the head-mounted temperature distribution recognition device 1. Such components can be realized by the components themselves shown in FIG. 1 or, for example, by the processor of the controller 20 executing a predetermined control program in cooperation with various hardware resources.

Referring to the figure, the sensor 40 includes, for example, a thermosensor 42 (corresponding to the reference numeral 42a in FIG. 1), a spatial recognition sensor 44, and a three-dimensional acceleration sensor 46, but is not limited thereto. . Thermosensor 42 may be, for example, a long wavelength infrared (LWIR) sensor module. The thermosensor 42 collects temperature data in a non-contact manner by detecting long-wavelength infrared rays emitted from a heat source in the target space using a pixel array, and outputs the data to the controller 20 .

Spatial awareness sensor 44 may include, for example, a visible light sensor and a depth sensor. The space recognition sensor 44 collects image data by detecting visible light observed in the target space using a pixel array and outputs the image data to the controller 20 in order to recognize a three-dimensional shape in the target space. Moreover, the space recognition sensor 44 can collect and output depth data (distance data) by measuring the depth direction (distance to an object) in the target space according to the TOF technique. Further, the spatial recognition sensor 44 may detect a user's gesture (for example, an air tap) as an input to the head-mounted temperature distribution recognition device 1 .

The three-dimensional acceleration sensor 46 detects, for example, a change in the movement of the user wearing the head-mounted temperature distribution recognition device 1 as three-dimensional acceleration data, and outputs the data to the controller 20 . The three-dimensional acceleration data from the three-dimensional acceleration sensor 46, together with the output from the spatial recognition sensor 44, is used to estimate the direction of the user's line of sight.

Although not shown in the figure, for example, the sensor 40 may include a sensor (eye tracker) that observes the movement of the user's eyes by emitting infrared light in order to identify the user's viewing direction.

As described above, the controller 20 is a computing device that centrally controls the operation of the head-mounted temperature distribution recognition device 1. In the figure, the functional configuration of the controller 20 includes an input control section 210, a temperature data processing section 220, a spatial orientation estimation section 230, a user input processing section 240, a data storage section 250, and an image data generation section 260. , an output control section 270, and a communication interface section 280 are shown.

The input control unit 210 includes an interface circuit that receives various data output from the sensor 40. That is, the input control unit 210 outputs the temperature data output from the thermosensor 40a to the temperature data processing unit 220, and also outputs the image data and/or depth data output from the spatial recognition sensor 44 to the spatial orientation estimation unit 230. Output to. The input control unit 210 also outputs three-dimensional acceleration data output from the three-dimensional acceleration sensor 46 to the spatial orientation estimation unit 230.

The temperature data processing unit 220 generates temperature distribution data based on the temperature data output from the thermosensor 40a. The temperature distribution data is, for example, data indicating the distribution of temperature levels in the target space. Thermography (temperature distribution map) is a two-dimensional or three-dimensional image of the temperature distribution based on temperature distribution data using color distribution and the like. As described later, the temperature data processing unit 220 can generate temperature distribution data based only on temperatures within a specific range, for example, according to predetermined setting conditions. The temperature data processing unit 220 generates temperature distribution data based on temperatures of 120 degrees or higher, for example, so that abnormal heat generation in electrical equipment can be easily distinguished visually. The temperature data processing section 220 outputs the generated temperature distribution data to the image data generation section 260. Further, the temperature distribution data generated by the temperature data processing section 220 may be recorded in the data storage section 250. The recorded temperature distribution data can be output to an external device via the communication interface unit 280, for example.

The spatial posture estimating unit 230 determines the user's posture, gaze direction, and/or direction in the target space based on image data and depth data output from the spatial recognition sensor 44, acceleration data output from the three-dimensional acceleration sensor 46, and the like. Or estimate or identify the gesture. For example, the spatial orientation estimating unit 230 estimates objects in the target space (e.g., floors, walls, ), calculate the displacement of the positional relationship, and estimate the user's posture and/or line of sight direction, that is, the posture and/or direction (for example, roll, pitch, and yaw) of the head-mounted temperature distribution recognition device 1. , and outputs this as spatial orientation data. Additionally, the spatial orientation estimation unit 230 estimates the user's gesture (for example, the movement of the user's finger (air tap), etc.) in front of the spatial recognition sensor 44, and outputs this as spatial orientation data to the image data generation unit 260. . Further, the spatial posture estimation section 230 outputs spatial posture data regarding the estimated user's posture, orientation, and/or gesture to the user input processing section 240 and the image data generation section 260. The spatial orientation data may include image data, depth data, three-dimensional acceleration data, and the like.

The user input processing unit 240 interprets the spatial orientation data related to gestures output from the spatial orientation estimation unit 230, and receives this as a user input. The user input processing unit 240 performs predetermined processing according to the received user input. As an example, the user input processing unit 240 generates a control command for displaying a menu on the display 30 in accordance with the user's air tap on the menu call icon displayed on the display 30, and outputs this to the image data generation unit 260. . As another example, the user input processing unit 240 executes various parameter setting processes according to the user's air tap on a setting icon among the menu icons displayed on the display 30.

The data storage unit 250 holds various data and setting data in the controller 20. As an example, the data storage section 250 can hold temperature distribution data output from the temperature data processing section 220. As another example, the data storage unit 250 can hold image data and depth data output from the spatial recognition sensor 44, acceleration data output from the three-dimensional acceleration sensor 46, and the like. Furthermore, as another example, the data storage unit 250 may hold parameters (setting data) regarding the operation settings of the controller 20.

The image data generation unit 260 generates temperature distribution data output from the temperature data processing unit 220, spatial orientation data output from the spatial orientation estimation unit 230, and/or various control commands output from the user input processing unit 240. Based on this, image data corresponding to the target space is generated. That is, the image data generation unit 260 generates image data in which the thermography indicated by the temperature distribution data is arranged in a superimposed manner with respect to the target space based on the attitude and/or orientation of the head-mounted temperature distribution recognition device 1 indicated by the spatial attitude data. generate.

That is, the image data generation unit 260 determines whether the thermal image based on the image data corresponding to the target space is hidden behind the object or by surrounding structures, based on the depth data to the object obtained by the space recognition sensor 44. The image data is generated so that the image data is displayed at a position slightly in front of the object. This allows the user to accurately and easily understand the thermal image.

Further, the image data generation unit 260 automatically adjusts the projection height and angle based on the spatial orientation data so that the projection height and angle are easy to see for the user, and generates image data. This allows the user to face the image data and clearly see the displayed content.

For example, the image data generation unit 260 detects a three-dimensional shape and/or edge of an object in the target space based on the image data and depth data, and superimposes thermography to match the three-dimensional shape and/or edge. It is also possible to perform image data generation processing for arranging images. For example, when a certain object (for example, electrical equipment, etc.) in the target space has a predetermined temperature, the image data generation unit 260 generates an image of the object so as to match the outline of the object that is visually recognized by the user. Image data is generated in which thermographs including temperature distribution corresponding to the temperature distribution are arranged in a superimposed manner. In other words, thermography based on temperature distribution data matches objects in the target space based on spatial orientation data. Further, the image data generation section 260 generates image data regarding various objects (for example, menu call icons, setting icons, etc.) based on control commands output from the user input processing section 240. Image data generation section 260 outputs the generated image data to output control section 270.

The output control unit 270 is an interface circuit that controls various output devices such as the display 30, the speaker 52, and the vibrator 54. For example, the output control unit 270 generates an image signal so that an image or video (hereinafter simply referred to as an “image”) based on the image data generated by the image data generation unit 260 is displayed on the display 30, Display control of the display 30 is performed. Further, the output control unit 270 causes the speaker 52 to sound or vibrates the vibrator 54 in accordance with the warning sound generation command output from the temperature data processing unit 220.

The display 30 displays an image based on the image data generated by the image data generation section 260 under the control of the output control section 270. In the present disclosure, for example, thermography, menu icons, and the like are displayed on the display 30 so as to be visible to the user.

The communication interface unit 280 includes an interface circuit for the head-mounted temperature distribution recognition device 1 to communicate with an external device (not shown) by wire and/or wirelessly. The communication interface unit 280 is configured in accordance with various communication interface standards such as USB, Bluetooth (registered trademark), or WiFi (registered trademark). For example, the head-mounted temperature distribution recognition device 1 communicates with a computing device via the communication interface section 280 by operating as a USB mass storage. Alternatively, the head-mounted temperature distribution recognition device 1 performs pairing with a computing device and performs communication using Bluetooth. As another example, the communication interface unit 280 is configured to receive radio waves transmitted from an RF tag.

FIG. 3 is a diagram for explaining an example of the operation mode of the head-mounted temperature distribution recognition device 1 according to an embodiment of the present invention. As shown in the figure, the operation modes of the head-mounted temperature distribution recognition device 1 may include, for example, a thermo monitoring mode 310, a display setting mode 320, a temperature setting mode 330, a photographing mode 340, a playback mode 350, and the like.

The thermo monitoring mode 310 is the main mode of the head-mounted temperature distribution recognition device 1, and is, for example, the mode that first operates when the head-mounted temperature distribution recognition device 1 is activated. In the thermo monitoring mode 310, the head-mounted temperature distribution recognition device 1 displays a thermography based on the temperature data detected by the thermo sensor 40a on the display 30 under the control of the controller 20, thereby providing the user with an MR space. I will provide a. FIG. 4 shows an example of a thermography screen displayed on the display 30 of the head-mounted temperature distribution recognition device 1 operating in thermo monitoring mode. In the example shown in the figure, a thermography based on temperature data detected by a thermosensor 40a is displayed in a thermography display area 401 provided approximately at the center of a thermography screen 400. Moreover, the temperature gauge 402 is an indicator that shows the temperature according to the color distribution shown by thermography. In the thermo monitoring mode 310, the user can display a menu window 501 as shown in FIG. 5 and select another operation mode by selecting the menu call icon 403 with a gesture such as an air tap. For example, the user can temporarily turn off thermo monitoring mode 310.

The display setting mode 320 is, for example, an operation mode for performing various display settings for the thermography screen displayed on the display 30. FIG. 6 shows an example of a display setting screen displayed on the display 30 of the head-mounted temperature distribution recognition device 1 in the display setting mode 320. In the display setting mode 320, the user can set the shape, size, etc. of the thermography display area 401 by air tapping or the like on the display setting screen 600 displayed on the display 30.

The temperature setting mode 330 is, for example, an operation mode for setting the temperature range shown in thermography. FIG. 7 is a diagram showing an example of a temperature setting screen displayed on the display 30 of the head-mounted temperature distribution recognition device 1 in the temperature setting mode 330. In the temperature setting mode 330, the user can set the display temperature range (lower limit value and/or upper limit value) in the thermography by inputting a gesture on the temperature setting screen 700 displayed on the display 30.

On the temperature setting screen 700, the user can set a temperature range value that causes the warning sound to be generated. For example, the user can set an alarm to sound when the temperature detected by the thermosensor 40a is equal to or higher than the upper limit operating temperature of the equipment to be inspected, 80 degrees. Further, even when the temperature detected by the thermosensor 40a exceeds a predetermined lower limit value, the pitch of the warning sound can be set to change according to the temperature. As yet another example, a sound may be output instead of or in addition to the warning sound. Since the abnormality that the user wants to find is a small part or area compared to the entire patrol target, emitting a warning sound in addition to the image allows the patrol to use the auditory sense in addition to the visual sense, which is highly effective in preventing oversights. Become.

The head-mounted temperature distribution recognition device 1 may be provided with a mode setting screen 800 on which the user can select a combination of ON/OFF of the thermography display and ON/OFF of the warning sound, as shown in FIG. . On the mode setting screen 800, the user can set temperatures lower than the temperature at which a warning sound is sounded (warning temperature) to be invisible (masked) in the thermography. For example, FIG. 9A shows a normal (no mask) thermography in thermo monitoring mode, while FIG. 9B shows a thermography with a temperature mask set. That is, in the same figure (b), the thermography is not displayed in the area below the warning temperature. This makes it possible to eliminate unnecessary images from the field of view, ensure visibility of the patrol route, and improve walking safety during inspection work. In addition, the sites to be patrolled are irregularly scattered with places where there is a risk of falling accidents or contact damage, such as steps, places where people can go up and down, and temporarily installed equipment under their feet or overhead. It will be possible to carry out patrol inspections while avoiding.

The shooting mode 340 may include, for example, a moving image shooting mode and a still image shooting mode. In the video shooting mode, the user can shoot a video using the image sensor. In the shooting mode, for example, a camera icon 1010 is displayed in a superimposed manner on the display 30, as shown in FIG. 10, in order to notify the user that shooting is in progress. Note that the captured moving image data and still image data are held in the data storage unit 250, for example.

The playback mode 350 is an operation mode in which images based on moving image data and still image data shot in the shooting mode are displayed on the display 30. In the playback mode, under the control of the controller 20, moving images and still images read out from the data storage unit 250 and played back are displayed on the display.

Note that settings in the various setting modes described above can be made using a keyboard, for example, from an external computing device communicably connected to the head-mounted temperature distribution recognition device 1, instead of inputting information on the screen displayed on the display 30. This may be done using a normal input device such as a computer or a mouse.

Next, an example of use of the head-mounted temperature distribution recognition device 1 of this embodiment will be described. Here, an example will be described in which a user wearing the head-mounted temperature distribution recognition device 1 inspects electrical equipment in a building.

A user routinely wears the head-mounted temperature distribution recognition device 1 and goes around a building while operating it in thermo monitoring mode. During the tour, the user focuses his/her line of sight on a location where electrical equipment is installed. For example, suppose that the user looks up at the ceiling in a hallway inside a building, as shown in FIG. 11(a). Then, the transmissive display 30 viewed by the user is provided with an MR space in which the thermography screen 400 is displayed in a superimposed manner, as shown in FIG. If electrical equipment such as cables is abnormally generating heat, the thermography screen 400 displayed on the display 30 will show a thermography showing a high temperature.

Moreover, abnormal heat generation at the power supply connection part can also be easily discovered using the head-mounted temperature distribution recognition device 1 of this embodiment. For example, FIG. 12(a) shows how the user visually checks the three-phase power supply connection section with the thermo monitoring mode turned off. In this state, it is difficult for the user to determine whether or not the three-phase power supply connection section is generating abnormal heat. On the other hand, when the user visually checks the three-phase power connection with the thermo monitoring mode turned on, the thermography screen 400 displayed on the display 30 shows the one-phase power connection (in the middle in the figure). The thermography will show a high temperature at the power supply connection (of the device). In other words, the head-mounted temperature distribution recognition device 1 detects the temperature of only one phase of the three-phase power supply connection section above a certain level (even if the temperature does not reach the upper limit of operation of the device and is below the allowable temperature value). By providing the user with an MR space in which the power connection of one phase exhibits a higher temperature than the other three phases, the user can easily recognize abnormal heat generation at the power connection of one phase. It becomes like this. In particular, being able to detect temperature differences (for example, 1°C to several degrees Celsius or more) between three-phase power supply connections in an MR space that includes thermal images makes it possible to detect slight overheating in the early stages when temperature differences begin to occur. This will make it possible to detect signs of a fire at an early stage, greatly contributing to the prevention of oversight.

As described above, according to the present embodiment, the user wearing the head-mounted temperature distribution recognition device 1 can recognize the target space in the viewing direction in real time as an MR space in which thermography is superimposed. become able to. Furthermore, since the head-mounted temperature distribution recognition device 1 is mounted on the user's head, the user can perform predetermined tasks without having both hands occupied. In particular, according to this embodiment, the user can work or move with both hands free, so even if the user accidentally falls down, he or she can immediately put his hands on his hands and take a passive stance to prevent a serious shock to his body. can be reduced.

Further, according to the present embodiment, the head-mounted temperature distribution recognition device 1 allows the user to set the display temperature of the thermography, and set the temperature range that is abnormal heat generation, and the temperature range that is normal temperature. can be set so that the thermography is invisible. Further, according to the present embodiment, the head-mounted temperature distribution recognition device 1 can turn off the thermography and set it to the transparent state while the warning function remains in operation, so the user can project the thermography only when necessary. As a result, visibility can be secured and work safety can therefore be improved.

Further, according to the present embodiment, the head-mounted temperature distribution recognition device 1 can record temperature data, image data, etc. detected by the sensor 40 in the data storage unit 250 as a log.

(Application example)
The head-mounted temperature distribution recognition device 1 of this embodiment can be modified and/or applied in various ways as described below.

(Application example 1)
The head-mounted temperature distribution recognition device 1 is connected to an external computing device by short-range communication such as Bluetooth, and is further connected to a management server (not shown) in a central monitoring room via a communication network. . In the central monitoring room, for example, an administrator checks the temperature data and/or image data transmitted in real time from the head-mounted temperature distribution recognition device 1, and sends information to the user wearing the head-mounted temperature distribution recognition device 1. Instructions can be given by voice or text messages.

Further, the management server in the central monitoring room stores temperature data and/or image data, etc. transmitted from the head-mounted temperature distribution recognition device 1 in a mass storage device. The management server performs AI learning by deep learning using the accumulated temperature data and/or image data as teaching data. For example, in a three-phase power connection, if one power connection is loose, the resistance will increase and overheating will occur. Therefore, if thermal image patterns in which three-phase power connections have the same appearance but a temperature difference in only one phase are accumulated as big data, AI learning can easily identify the power connection where the temperature is rising. be able to identify.

In particular, at electrical equipment sites, many devices with similar shapes are lined up side by side, making it easy to overlook abnormalities. By recognizing areas with temperature differences from thermal image data, it is expected to detect overheating abnormalities at an early stage, improving the effectiveness of preventing oversights.

(Application example 2)
For example, markers are installed inside a building so that electrical equipment (suspected locations) to be inspected within the building can be periodically monitored without being overlooked. The marker is, for example, a bar code or an RF tag, and records location information of electrical equipment. A user who wears the head-mounted temperature distribution recognition device 1 can read a barcode using the sensor 40 or receive radio waves from an RF tag using the communication interface section 280, for example. , under the control of the controller 20, acquires position information of the electrical equipment associated with the marker. The head-mounted temperature distribution recognition device 1 guides the user by displaying on the display 30 the position or direction in which the electrical equipment associated with the marker is installed according to the acquired position information. This allows the user to easily know the location of electrical equipment in the building, eliminating oversights and leading to enhanced monitoring.

Further, the head-mounted temperature distribution recognition device 1 can transmit the acquired position information to the management server together with the temperature data and/or image data. The management server can store the received temperature data and/or image data in a storage device in association with location information. For example, the management server may display the accumulated temperature data and/or image data in association with geographical information within the building.

Alternatively, the head-mounted temperature distribution recognition device 1 may be configured to record the XYZ coordinates and photographing direction of the point where the thermo image was photographed based on the spatial orientation data. For example, the head-mounted temperature distribution recognition device 1 sets the point where the power of the device is turned on as the coordinate origin, and calculates and records the current coordinates as the user moves/moves. Further, the head-mounted temperature distribution recognition device 1 can record the direction and inclination of the user looking directly at the object. This allows the user to wear the head-mounted temperature distribution recognition device 1 and perform on-site inspection work, and then utilize these data when creating an inspection work record at the office. You will be able to do this.

Each of the embodiments described above is an illustration for explaining the present invention, and the present invention is not intended to be limited only to these embodiments. The present invention can be implemented in various forms without departing from the gist thereof.

For example, steps, acts, or functions in the methods disclosed herein may be performed in parallel or in a different order unless the results are inconsistent. The steps, acts, and functions described are provided by way of example only, and some of the steps, acts, and functions may be omitted or combined with each other without departing from the spirit of the invention. It is also possible to add other steps, actions, or functions.

Further, although various embodiments are disclosed in this specification, specific features (technical matters) in one embodiment may be added to other embodiments while improving them as appropriate, or Certain features in the embodiments may be replaced and such forms are within the scope of the invention.

1...Head mounted temperature distribution recognition device 10...Head mount section 20...Controller 210...Input control section 220...Temperature data processing section 230...Spatial attitude estimation section 240...User input processing section 250...Data storage section 260...Image data generation Section 270...Output control section 280...Communication interface section 30...Display 40...Sensor 40a, 42...Thermo sensor 40b...Visible light sensor 44...Spatial recognition sensor 46...Three-dimensional acceleration sensor 52...Speaker 54...Vibrator

Claims (9)

A head-mounted temperature distribution recognition device,
A system comprising the head-mounted temperature distribution recognition device and a management server connected via a communication network,
The head-mounted temperature distribution recognition device includes:
a head mount part that can be attached to the user's head;
A thermosensor for detecting at least infrared rays emitted from a heat source in a target space, which is arranged so as to substantially match the direction of the line of sight of a user wearing the head mount unit, and a three-dimensional shape in the target space. a sensor including a spatial recognition sensor for;
a display placed in front of the user's line of sight;
a controller that controls the sensor and the display;
The controller includes:
Generate image data including thermography in the target space based on first data output from the thermosensor and second data output from the space recognition sensor, and based on the generated image data. controlling an image to be displayed on the display;
Accepting an air tap gesture of the user in front of the spatial recognition sensor as a user input, and controlling the display so that a predetermined menu is displayed in accordance with the accepted user input;
When the sensor reads a marker that records location information of the electrical equipment, control is performed so that the location information of the electrical equipment is acquired and the position or direction in which the electrical equipment is installed is displayed on the display. and transmitting the location information to the management server together with the first data and/or the image data,
The management server is
Based on the temperature data and image data transmitted in real time from the head-mounted temperature distribution recognition device, a predetermined temperature difference occurs between a plurality of electrical components of the same type, and the electrical component exhibiting a high temperature is identified as a heating abnormality. determining that the first data and/or the image data received from the head-mounted temperature distribution recognition device are stored in a storage device in association with the position information; displaying the first data and/or the image data in association with geographical information within the building;
system. The controller includes:
a temperature data processing unit that generates temperature distribution data based on first data output from the thermosensor;
a spatial orientation estimation unit that generates spatial orientation data based on second data output from the spatial recognition sensor;
an image data generation unit that generates the image data including the thermography so that the thermography based on the temperature distribution data matches an object in the target space based on the spatial orientation data;
Equipped with
The system of claim 1. The controller is configured to be able to set a display temperature range in the thermography according to the air tap gesture for the predetermined menu,
The image data generation unit generates the image data including the thermography according to the set display temperature range,
The system according to claim 2. The controller is configured to notify the user when the temperature indicated by the first data output from the thermosensor is equal to or higher than a predetermined warning temperature.
The system according to claim 3. The image data generation unit generates the image data so that a temperature lower than the predetermined warning temperature becomes invisible in the thermography.
The system according to claim 4. The thermosensor is removably attached to the head mount,
A system according to any one of claims 1 to 5. The sensor includes a pair of thermosensors arranged to correspond to the binocular disparity of the user so that the user perceives a three-dimensional thermography.
A system according to any one of claims 1 to 6. A head-mounted temperature distribution recognition device,
A system comprising the head-mounted temperature distribution recognition device and a management server connected via a communication network,
The head-mounted temperature distribution recognition device includes:
a head mount part that can be attached to the user's head;
A thermosensor for detecting at least infrared rays emitted from a heat source in a target space, which is arranged so as to substantially match the direction of the line of sight of a user wearing the head mount unit, and a three-dimensional shape in the target space. a sensor including a spatial recognition sensor for;
a display placed in front of the user's line of sight;
a controller that controls the sensor and the display;
The controller includes:
Generate image data including thermography in the target space based on first data output from the thermosensor and second data output from the space recognition sensor, and based on the generated image data. controlling an image to be displayed on the display;
When the sensor reads a marker that records location information of the electrical equipment, control is performed so that the location information of the electrical equipment is acquired and the position or direction in which the electrical equipment is installed is displayed on the display. and transmitting the location information to the management server together with the first data and/or the image data,
The management server is
Based on the temperature data and image data transmitted in real time from the head-mounted temperature distribution recognition device, a predetermined temperature difference occurs between a plurality of electrical components of the same type, and the electrical component exhibiting a high temperature is recognized as having a heating abnormality. determining that the first data and/or the image data received from the head-mounted temperature distribution recognition device are stored in a storage device in association with the position information; displaying the first data and/or the image data in association with geographical information within the building;
system. the marker is a barcode or an RF tag;
The system according to claim 1 or 8 .

Images