JP7637928B2 - Helmet-compatible display system, display control method, helmet, and program - Google Patents

Description

The present invention relates to a helmet-compatible display system, a display control method, a helmet, and a program.

There is known technology that provides a helmet with a wearable terminal that can be positioned in front of the worker's eyes, allowing the worker wearing the helmet to view camera footage and data images displayed on the wearable terminal's display (Patent Document 1).

JP 2018-119241 A

In an environment where a helmet that displays images to the wearer is used, it is preferable to improve the convenience of the user by utilizing the helmet.

The present invention was made in consideration of these circumstances, and aims to improve user convenience by utilizing a helmet that displays images toward the wearer.

One aspect of the present invention for solving the above-mentioned problems is a helmet-compatible display system that includes explosion-proof glasses provided on a helmet, a display unit provided on the explosion-proof glasses to display an image to a user when the user wears the explosion-proof glasses, a plurality of imaging devices provided on the helmet at a predetermined distance from each other and each configured to capture an image of a predetermined direction outside the helmet when the user wears the helmet, an information processing unit that performs predetermined information processing using the results of measuring the distance to an object in a captured image by utilizing the parallax between a plurality of captured images obtained by capturing images by each of the plurality of imaging devices, and a display control unit that causes an image corresponding to the information processing by the information processing unit to be displayed on the display unit.

Also, one aspect of the present invention is a display control method for a display system corresponding to a helmet, the display system including explosion-proof glasses, a display unit provided on the explosion-proof glasses to display an image toward a user when the user wears the explosion-proof glasses, and a plurality of imaging devices provided on the helmet at a predetermined distance from each other, each of which is provided to capture an image of a predetermined direction outside the helmet when the user wears the helmet, the display control method including an information processing step in which an information processing unit performs predetermined information processing using a result of measuring a distance to an object in a captured image by utilizing parallax between a plurality of captured images obtained by capturing images by each of the plurality of imaging devices, and a display control step in which a display control unit causes the display unit to display an image corresponding to the information processing performed by the information processing step.

Another aspect of the present invention is a helmet that includes explosion-proof glasses, a display unit provided on the explosion-proof glasses to display an image to a user when the user wears the explosion-proof glasses, a plurality of imaging devices provided on a helmet at a predetermined distance from each other and each of which is arranged to capture an image of a predetermined direction outside the helmet when the user wears the helmet, and a display control unit that causes the display unit to display an image according to a predetermined information processing that uses the results of measuring the distance to an object in the captured image by using the parallax between a plurality of captured images obtained by each of the plurality of imaging devices.

Another aspect of the present invention is a program for causing a computer in a helmet, which includes explosion-proof glasses, a display unit provided on the explosion-proof glasses to display an image to a user when the user wears the explosion-proof glasses, and a plurality of imaging devices provided on the helmet at a predetermined distance from each other, each of which is provided to capture an image of a predetermined direction outside the helmet when the user wears the helmet, to function as a display control unit that causes the display unit to display an image corresponding to a predetermined information processing that uses the results of measuring the distance to an object in the captured image by utilizing the parallax between a plurality of captured images obtained by each of the plurality of imaging devices.

As described above, the present invention has the effect of improving user convenience by utilizing a helmet that displays images toward the wearer.

1 is a diagram showing an example of the appearance of a helmet according to an embodiment of the present invention; 1 is a diagram showing an example of the configuration of a helmet according to an embodiment of the present invention;

The helmet of the present embodiment will be described below. The helmet of the present embodiment is, for example, worn by a worker at a construction site.
An example of the configuration of a helmet 1 according to this embodiment will be described with reference to Fig. 1 and Fig. 2. Fig. 1 shows an example of the appearance of the helmet 1 according to this embodiment. In the figure, a state in which the helmet 1 is worn by a worker W (an example of a user) is shown. Fig. 2 shows an example of the functional configuration of the helmet 1.

As shown in FIG. 1 , a helmet 1 of this embodiment has an outer shape including a main body portion 11 and a visor 12 .
As shown in FIG. 2, the helmet 1 of this embodiment includes a headlight 20, explosion-proof goggles 30, a camera 40 (40-L, 40-R), a bone conduction headset 50, a battery 60, a power supply circuit 65, a control unit 100, a communication unit 110, and a memory unit 120.

As shown in FIG. 1, the headlight 20 is arranged to illuminate the area ahead of the worker W when the headlight 20 is worn by the worker W.

As shown in FIG. 1, the left and right ends of the temples 31 of the explosion-proof goggles 30 are connected to holders 13 provided on the helmet 1. The holders 13 are rotatable at the portions that hold the connected temples 31. This allows the worker W to change the state of the explosion-proof goggles 30 between covering the eyes of the worker W as shown in the figure, and storing them inside the helmet 1 by pulling them up from the state shown in the figure.

As shown in FIG. 2, the explosion-proof goggles 30 include a display unit 35. The display unit 35 displays an image directed at the worker W when the worker W is wearing the explosion-proof goggles 30 so that they cover his or her eyes.

There is no particular limitation on the display device that the display unit 35 is compatible with. The display device that the display unit 35 is compatible with may be, for example, one that displays an image on a lens, or one that uses retinal projection or the like. The displayed image may be planar, or may be one that is projected three-dimensionally in the field of view of the worker W.

By providing the display unit 35 in this manner, the worker W can observe the surroundings and the displayed image through the lenses of the explosion-proof goggles 30 while wearing the explosion-proof goggles 30 to cover his or her eyes.

1, the cameras 40 (40-L, 40-R) are provided in a mutually left-right positional relationship and are provided so as to capture an image of the front according to the line of sight of the worker W wearing the helmet 1. The camera 40-L is provided on the front left side of the frame of the explosion-proof goggles 30, and the camera 40-R is provided on the front right side of the frame of the explosion-proof goggles 30.
The two cameras 40 thus provided can function as a stereo camera capable of calculating the distance to an object in a captured image by utilizing the parallax between the left and right cameras.

In FIG. 1, an example is shown in which the camera 40 is attached to the explosion-proof goggles 30, but the camera 40 may be installed, for example, on the left or right side of the main body 11 of the helmet 1 so as to capture an image of the front according to the line of sight of the worker W wearing the helmet 1.

1, the bone conduction headset 50 is a headset in which a bone conduction earphone 52 and a bone conduction microphone 53 are provided on a main body 51. The bone conduction headset 50 is attached to the helmet 1 by a connector 55.
When the bone conduction headset 50 in the figure is hung on the ears of the worker W, the bone conduction earphones 52 come into contact with the temples of the worker W and the bone conduction microphone 53 comes into contact with the cheek of the worker W. This allows the worker W to hear sounds from the bone conduction earphones 52 via bone conduction, and also allows the bone conduction microphone 53 to pick up sounds that the worker W makes.
By hearing voices and collecting spoken voices through bone conduction in this way, the worker W can clearly hear the other person's voice and can clearly hear his own voice even in a loud environment such as a construction site. In other words, by using the bone conduction headset 50, the worker W can smoothly converse with others regardless of the surrounding noise.

In addition, when operating an application executed by the control unit 100, the worker may perform operations by voice input by speaking a command language into the bone conduction microphone 53.

The battery 60 stores power to be supplied to devices that operate on power in the helmet 1. The battery 60 may be, for example, a primary battery, or a secondary battery such as a lithium ion battery. The type, shape, etc. of the battery 60 are not particularly limited.
The battery 60 is provided so as to be attached to a predetermined position in the helmet 1. The battery 60 may be detachably attached to the helmet 1. Furthermore, if the battery 60 is a secondary battery, the battery 60 may be adapted to be capable of being charged while being attached to the helmet 1.

The power supply circuit 65 inputs the power stored in the battery 60, and outputs a power supply voltage generated from the input power to various devices provided in the helmet 1. In this embodiment, the devices to which the power supply voltage is supplied by the power supply circuit 65 include the headlight 20, the display unit 35, the camera 40, and the bone conduction headset 50. The devices to which the power supply voltage is supplied by the power supply circuit 65 also include a computer that realizes the functions of the control unit 100, the communication unit 110, etc. The computer corresponding to the control unit 100, the communication unit 110, etc. is also provided so as to be housed inside the helmet 1.
Also, for example, the power supply circuit 65 may be capable of inputting DC or AC power from an external source to charge the battery 60 .

The control unit 100 executes various controls in the helmet 1. The control unit 100 controls various devices in the helmet 1 (headlight 20, display unit 35, camera 40, bone conduction headset 50, etc.). The control unit 100 also executes control related to communication by the communication unit 110.

The communication unit 110 communicates with the outside wirelessly. The wireless communication supported by the communication unit 110 is not particularly limited, and may be, for example, a wireless LAN. The communication unit 110 may be capable of short-distance wireless communication using, for example, Bluetooth (registered trademark).
The storage unit 120 stores various types of information used by the control unit 100 .

The helmet 1 configured as described above is capable of transmitting and receiving voice by wireless communication via the communication unit 110. This allows the worker W to use the bone conduction headset 50 to converse with other people, such as other workers or managers at headquarters.

In addition, the helmet 1 having the above configuration can display images on the explosion-proof glasses 30 equipped with a display unit 35 as smart glasses, under the control of the control unit 100.
For example, the control unit 100 can display information reflecting the results of a predetermined analysis of the captured image obtained by capturing an image with the camera 40 .
Furthermore, the control unit 100 can display information acquired from an external server or a terminal information processing device through communication by the communication unit 110 .

In addition, the camera 40 (40-L, 40-R) of this embodiment can be used as a stereo camera imaging device. Therefore, for example, a system may be constructed that measures the position of a specific object at a construction site by using an image obtained by imaging a specific object with the camera 40 as a stereo camera. Such a system may be configured to be completed in the helmet 1, for example, by having the control unit 100 input the captured image and perform measurement. Alternatively, the helmet 1 may transmit the captured image to an external device, and the external device may measure the position of the object. In this case, the external device may be, for example, an external mobile terminal (for example, a smartphone, a tablet terminal, etc.) held by the worker W, a terminal such as a computer installed in an office, or a server.
In addition, in each of the above systems, an image reflecting the measurement results of the object may be displayed to the worker W on the display unit 35 of the explosion-proof goggles 30.

Specifically, for example, at a construction site where a reinforced concrete structure is being constructed, rebars are arranged before the concrete is poured. Rebar arrangement inspection is an inspection carried out to confirm and prove that the rebars are arranged correctly before the arranged rebars are embedded in the concrete by pouring the concrete. In a rebar arrangement inspection, for example, the position, diameter, pitch, number, etc. of the arranged rebars are inspected.

The helmet 1 of this embodiment is configured to be capable of performing processing corresponding to reinforcement inspection. The control unit 100 of this embodiment includes an information processing unit 101 and a display control unit 102 as functional units corresponding to reinforcement inspection. The information processing unit 101 executes information processing (reinforcement inspection processing) corresponding to the reinforcement inspection. The display control unit 102 causes the display unit 35 to display an image corresponding to the reinforcement inspection processing by the information processing unit 101.

The following describes a workflow of a bar arrangement inspection according to the present embodiment, and an example of a bar arrangement inspection process according to the workflow. In the following description, reinforcing bars arranged at a construction site may also be referred to as "arranged reinforcing bars."
The worker W wearing the helmet 1 stands at an inspection position determined in accordance with the range of the arranged rebars to be inspected among the arranged rebars, so that the arranged rebars in the range to be inspected are imaged by the camera 40. At this time, the image captured by the camera 40 is displayed on the display unit 35 as a through image by the display control unit 102. This allows the worker W to adjust the direction of the helmet 1, etc., while looking at the displayed through image so that the range of the arranged rebars to be inspected is appropriately within the imaging range of the camera 40.

The worker W performs an operation to instruct the camera 40 to take an image for the reinforcing bar inspection (photographing instruction operation) with the range of the reinforcing bars to be inspected included in the imaging range of the camera 40. The photographing instruction operation may be an operation on an operator for photographing instruction operation provided at a predetermined position on the helmet 1, or may be an operation by voice input using the bone conduction microphone 53.
The information processing unit 101 acquires, as an inspection image, the captured image obtained by the camera 40 at the time when the shooting instruction operation was performed. The inspection images acquired by the information processing unit 101 are two captured images obtained by capturing images using the two cameras 40-L and 40-R, respectively.

The information processing unit 101 utilizes the parallax between the two acquired inspection images to measure the distance to the arranged reinforcing bars in the inspection images.
The information processing unit 101 determines the reinforcement state, such as the position of the arranged rebars, the diameter of the arranged rebars, the pitch (spacing) between the arranged rebars, the number of arranged rebars, etc., based on the measured distance between the arranged rebars and the position where the worker W is standing to take the image.
The information processing unit 101 also judges whether the identified reinforcement state is appropriate according to the design (reinforcement evaluation). At this time, the information processing unit 101 may perform the reinforcement evaluation by comparing the reinforcement design contents indicated by the reinforcement design data stored in the storage unit 120 with the identified reinforcement state. If the reinforcement evaluation judges that the identified reinforcement state is inappropriate, the information processing unit 101 identifies the inappropriate parts of the reinforcement state (i.e., the reason why the reinforcement evaluation is inappropriate).

The information processing unit 101 generates an image (reinforcement inspection image) reflecting the specified result of the reinforcement state. The display control unit 102 causes the display unit 35 to display the reinforcement inspection image generated by the information processing unit 101.
The worker W can check, for example, the specified reinforcement state by looking at the reinforcement inspection image displayed on the display unit 35. Also, the worker W can check whether the arranged reinforcing bars are appropriate or not by looking at the reinforcement inspection image. Also, by looking at the reinforcement inspection image, the worker W can check which part of the arranged reinforcing bars is not appropriate if the arranged reinforcing bars are not appropriate.

For example, when worker W is trying to fit the range of the arranged rebars to be inspected within the imaging range of camera 40, the information processing unit 101 may detect rebars that are obstructed at the rear side of the rebars on the near side (camera 40 side) of the captured image (detection of obstructed rebars).
For this reason, the information processing unit 101 acquires the captured image obtained by the camera 40 as an image to be used for detecting the shielded rebar. The information processing unit 101 detects the rear rebar (shielded rebar) that is shielded by the front rebar using the acquired captured image. For example, the information processing unit 101 may detect, among the rear rebars, a rear rebar that is not present in one captured image but is present in the other captured image as the shielded rebar.
When the information processing unit 101 detects a shielded rebar, it causes the display unit 35 to display an image (shielded rebar notification image) that notifies the worker W that the shielded rebar has been detected. Although not shown in the figure, the shielded rebar notification image may be an image that notifies the worker W that a shielded rebar has been detected and indicates the position of the detected shielded rebar.
A worker W who sees the shielded rebar warning image can adjust it so that the shielded rebar is imaged without being shielded, for example, by changing the imaging direction of the camera 40 or slightly shifting his or her standing position.

The information processing unit 101 may store data reflecting the results of the reinforcement evaluation (which may be in a file format, for example) in the memory unit 120. Alternatively, the information processing unit 101 may transmit the data reflecting the results of the reinforcement evaluation from the communication unit 110 to an external terminal or an external server.

With the helmet 1 of this embodiment configured as described above, the worker W can perform a reinforcement inspection by simply wearing the helmet 1, without using a mobile terminal such as a tablet terminal or smartphone equipped with a reinforcement inspection application. In other words, the convenience of the worker W during work at the construction site is improved.

The information processing unit 101 may be equipped with an information processing device such as a server or an external information processing terminal outside the helmet 1. In this case, the helmet 1 functions as, for example, a thin client terminal. That is, the control unit 100 of the helmet 1 in this case transmits the captured image obtained by capturing an image with the camera 40 from the communication unit 110 to the external information processing device. The information processing device executes the reinforcement inspection process and transmits information on the results of the executed reinforcement inspection process to the helmet 1. The display control unit 102 of the helmet 1 displays the received information on the results of the reinforcement inspection process on the display unit 35. In addition, the helmet 1 of this embodiment can be used not only for reinforcement inspection but also for other inspections using captured images. For example, it can be used to inspect the condition of objects, such as for remote attendance when an inspector remotely checks the construction status of objects other than reinforcing bars at a construction site.

The helmet 1 of the present embodiment is not limited to being used by a worker W at a construction site. For example, the helmet 1 of the present embodiment may be used by a worker who performs inspection, maintenance, etc. at high places, or may be used for disaster prevention, etc.
Furthermore, the process executed by the information processing unit 101 and the image that the display control unit 102 displays on the display unit 35 may be appropriately changed depending on the application of the helmet 1. For example, when the helmet 1 is used for disaster prevention, the information processing unit 101 may acquire disaster information received from a server or the like that provides disaster information on a network via the communication unit 110, and the display control unit 102 may display the acquired disaster information on the display unit 35. Furthermore, in this case, the information processing unit 101 may analyze an image captured by the camera 40 to detect a dangerous place, and the display control unit 102 may display a warning to a user wearing the helmet 1 on the display unit 35.

A program for realizing the functions of the above-mentioned control unit 100, etc. may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to perform the processing of the above-mentioned control unit 100, etc. Here, "reading a program recorded on a recording medium into a computer system and executing it" includes installing the program into a computer system. The "computer system" here includes hardware such as an OS and peripheral devices. The "computer system" may also include multiple computer devices connected via a network including a communication line. The recording medium also includes a recording medium provided internally or externally that can be accessed from a distribution server in order to distribute the program.

1 Helmet, 20 Headlight, 30 Explosion-proof goggles, 35 Display unit, 40 (40-L, 40-R) Camera, 50 Bone conduction headset, 52 Bone conduction earphone, 53 Bone conduction microphone, 60 Battery, 65 Power supply circuit, 100 Control unit, 101 Information processing unit, 102 Display control unit, 110 Communication unit, 120 Memory unit

Claims (7)

Explosion-proof goggles that can be attached to the helmet,
A display unit provided on the explosion-proof glasses to display an image toward a user when the user wears the explosion-proof glasses;
A plurality of imaging devices are provided on the helmet at a predetermined distance from each other, and each imaging device is provided so as to capture an image of a predetermined direction outside the helmet when the user wears the helmet;
an information processing unit that performs predetermined information processing using a result of measuring a distance to an object in a captured image by utilizing a parallax between a plurality of captured images obtained by capturing images by each of the plurality of imaging devices;
a display control unit that causes an image corresponding to the information processing by the information processing unit to be displayed on the display unit ,
The information processing unit executes information processing as a reinforcing bar arrangement inspection based on a distance measured using reinforcing bars arranged at a construction site as an object,
The display control unit causes the display unit to display a reinforcement inspection image reflecting a result of the reinforcement inspection.
Helmet compatible display system. The information processing unit, when acquiring the captured images used in the reinforcement inspection, detects a rear-side shielded reinforcing bar that is shielded by a front-side reinforcing bar by referring to each of the plurality of captured images,
The display control unit causes a shielded reinforcing bar notification image, which notifies the user that the shielded reinforcing bar has been detected, to be displayed on the display unit.
2. The helmet compatible display system of claim 1. The helmet-compatible display system according to claim 1 or 2 , wherein the information processing unit is provided in a helmet. The helmet-compatible display system according to claim 1 or 2 , wherein the information processing unit is provided in an information processing device outside the helmet. The helmet-compatible display system according to claim 1 , wherein the plurality of imaging devices are provided in the explosion-proof goggles. Explosion-proof goggles,
A display unit provided on the explosion-proof glasses to display an image toward a user when the user wears the explosion-proof glasses;
A display control method for a display system corresponding to a helmet including a plurality of imaging devices provided on the helmet at a predetermined distance from each other, each imaging device being provided so as to capture an image of a predetermined direction outside the helmet when a user wears the helmet, comprising:
an information processing step in which an information processing unit performs a predetermined information processing using a result of measuring a distance to an object in a captured image by utilizing a parallax between a plurality of captured images obtained by capturing images by each of the plurality of imaging devices;
a display control step of causing a display control unit to display an image corresponding to the information processing performed by the information processing step on the display unit;
The information processing step performs information processing as a reinforcing bar arrangement inspection based on a distance measured using reinforcing bars arranged at a construction site as an object,
The display control step causes the display unit to display a reinforcement inspection image reflecting a result of the reinforcement inspection.
Display control method. Explosion-proof goggles,
A display unit provided on the explosion-proof glasses to display an image toward a user when the user wears the explosion-proof glasses;
A plurality of imaging devices are provided on the helmet at a predetermined distance from each other, and each imaging device is provided so as to capture an image of a predetermined direction outside the helmet when the user wears the helmet;
a display control unit that causes the display unit to display an image corresponding to a predetermined information processing that uses a result of measuring a distance to an object in a captured image by using a parallax between a plurality of captured images obtained by capturing images by each of the plurality of imaging devices ,
The display control unit causes the display unit to display a reinforcing bar inspection image reflecting a result of information processing as a reinforcing bar inspection based on a distance measured using reinforcing bars arranged at a construction site as an object.
helmet.

Images