JP7480241B2 - Tracking device, plotting system, plotting method and program - Google Patents

Description

The present invention relates to a tracking device, a plotting system, a plotting method and a program, and in particular to a technology for electronically managing marking locations made with a writing implement.

Traditionally, when inspecting structures such as tunnels, bridges, and buildings, people would identify abnormalities by visually inspecting, tapping, and palpating the walls, marking the abnormalities with a writing implement, sketching the marked locations, and incorporating the results into drawings based on the sketch. For example, in the case of a tunnel inspection, the abnormalities on the lining surface would be marked with chalk, the marking locations would be sketched on paper, and the sketch would be traced using CAD to create a lining development drawing.

However, conventional methods require the user to sketch the marking locations, take the sketch home, and digitize the information before transcribing it. This type of transcription work is extremely cumbersome and time-consuming. In addition, because the transcription work is done manually, there is a possibility that mistakes or errors may occur. Therefore, if the coordinates of the abnormality areas could be obtained electronically using some method, the transcription work could be eliminated, improving efficiency and accuracy.

Patent documents 1 and 2 describe a system that photographs the inner walls of a tunnel and identifies abnormalities by analyzing the photographed images.

International Publication No. 2019/021719 JP 2017-211314 A

However, the technique of identifying abnormalities by image analysis, as in the techniques described in Patent Documents 1 and 2, has the problem of requiring huge amounts of data to be processed. In addition, there is a possibility that abnormalities that have traditionally been discovered not only by visual information but also by tapping or palpation may be overlooked. Therefore, it would be convenient to be able to electronically grasp the coordinate data of the markings made with a writing implement while still maintaining the process of manually recognizing abnormalities.

The present invention has been made to solve these problems, and aims to provide a tracking device, plotting system, plotting method, and program that can electronically manage marking locations made with a writing instrument.

One form of the present invention is a tracking device having a marking unit for marking an object, a detection unit for detecting when the marking has started or ended, and a position information generation unit for generating position information and transmitting it to a plotting device, wherein the position information generation unit starts at least one of generating or transmitting position information when the detection unit detects the start of the marking.
In another form of the present invention, the detection unit is detachable from the marking unit and is provided with a sensor at the point of contact with the marking unit or the user's body, and the sensor detects the pressure change when the marking is performed by the marking unit, thereby detecting the start or end of the marking.
In another aspect of the invention, the position information is coordinates with respect to the plotting device as the origin.
One form of the present invention is a tracking device worn on the upper limb of a user who marks an object, the tracking device having a detection unit that detects when the marking has started or ended, and a position information generation unit that generates position information and transmits it to a plotting device, the position information generation unit starting at least one of generating or transmitting position information when the detection unit detects the start of the marking.
In another aspect of the present invention, the detection unit detects the start or end of the marking by detecting a change in sensor output information including sound, speed or acceleration, or input of user information including a tap, gesture or voice.
In another aspect of the present invention, the position information is coordinates with a marker placed in real space as the origin, and the position information generation unit recognizes the position of the marker by a camera.
One form of the present invention is a tracking device worn on the head of a user who marks an object, the tracking device having a detection unit that detects when the marking has started or ended, and a position information generation unit that generates position information of a tool used to mark the object and transmits it to a plotting device, the position information generation unit starting at least one of generating and transmitting position information when the detection unit detects the start of the marking.
In another aspect of the present invention, the detection unit detects the start or end of the marking by detecting input of user information including a gesture or a voice.
In another aspect of the invention, the location information is expressed in a coordinate system of the tracking device.
In another aspect of the present invention, the position information generation unit terminates at least one of the generation and transmission of the position information when the detection unit detects the end of the marking.
One form of the present invention is a plotting system comprising the tracking device and a plotting device, the plotting device having a location information receiving unit that receives the location information, a location information storage unit that stores the received location information, and a plotting unit that displays the stored location information.
One form of the present invention is a plotting method comprising a marking step of marking an object, a detection step of detecting that the marking has started or ended, and a position information generation step of generating position information and transmitting it to a plotting device, wherein in the position information generation step, at least one of the generation and transmission of position information is initiated when the detection unit detects the start of the marking.
One form of the present invention is a program for causing a computer to execute a detection step of detecting that marking has started or ended, and a position information generation step of generating position information and transmitting it to a plotting device, wherein in the position information generation step, the program starts at least one of generating or transmitting position information when the detection unit detects the start of the marking.

The present invention provides a tracking device, plotting system, plotting method, and program that can electronically manage marking locations made with a writing implement.

FIG. 1 is a block diagram showing a configuration of a plot system 100. FIG. 1 is a block diagram showing a configuration of a plot system 100 according to a first embodiment. 1 is a diagram showing an example of the appearance of a tracking device 1 in embodiment 1. FIG. 11 is a block diagram showing a configuration of a plotting system 100 according to a second embodiment. FIG. 11 is a diagram showing an example of the appearance of a plot system 100 in embodiment 2. FIG. 11 is a diagram showing an example of the appearance of a plot system 100 in embodiment 2. FIG. 11 is a block diagram showing a configuration of a plotting system 100 according to a third embodiment. FIG. 13 is a diagram showing an example of the appearance of a plot system 100 in embodiment 3. FIG. 13 is a diagram showing an example of the appearance of a plot system 100 in embodiment 3.

The following describes an embodiment of the present invention with reference to the drawings. First, the configuration of a plotting system 100 according to an embodiment of the present invention will be described with reference to FIG. 1. The plotting system 100 includes a tracking device 1 that acquires the trajectory of a drawing or writing made by a writing implement as time-series coordinate data, and a plotting device 2 that plots the time-series coordinate data.

<First embodiment>
2 is a block diagram showing the configuration of a plotting system 100 according to the first embodiment. In the first embodiment, a tracking device 1 is configured by attaching a module having a position information acquisition function and a communication function to a writing instrument or the like for marking, and a plotting device 2 is provided outside the tracking device 1 to manage the coordinate data of the marking location.

The tracking device 1 has a marking unit 11, a position information generating unit 13, and a detection unit 15.

The marking unit 11 is a tool for marking an object, and includes, for example, writing implements such as chalk, pens, pencils, and markers, as well as inkjet printers, sprays, stamps, etc. The inspector uses the marking unit 11 to directly mark abnormalities on the walls of structures such as tunnels, bridges, and buildings.

The location information generating unit 13 acquires the location information of the tracking device 1 and transmits it to the plotting device 2.

For example, the position information generating unit 13 may receive and analyze signals emitted by the plot device 2 from multiple radio wave sources, and detect the relative position with the plot device 2 as the origin by three-point positioning or the like. In this case, for example, signals such as Wi-Fi, Bluetooth (registered trademark), electromagnetic waves including light, or sound waves can be used. Alternatively, the position information generating unit 13 may detect the relative position with the plot device 2 as the origin by autonomous navigation using an acceleration sensor, a gyro sensor, a magnetic sensor, or the like. According to these methods, it is possible to grasp the position information of the tracking device 1 even in an indoor environment as long as the plot device 2 is in a position where it can be seen. Alternatively, the position information generating unit 13 may obtain its own position information by receiving and analyzing positioning signals transmitted from multiple sources. One typical method is the Global Navigation Satellite System (GNSS), such as the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), Galileo, and the Quasi-Zenith Satellite System (QZSS).

The position information acquired by the position information generating unit 13 is typically three-dimensional coordinate data. The position information generating unit 13 can acquire the position information continuously, that is, at a predetermined time interval over a predetermined period of time. The position information generating unit 13 transmits the acquired position information to the plotting device 2 in real time or at a predetermined timing using any wireless or wired communication method.

The detection unit 15 detects the start and end of marking by the marking unit 11. For example, if the marking unit 11 is a writing implement such as chalk or a stamp, the detection unit 15 is equipped with a pressure sensor and can detect the start and end of marking by detecting pressure changes when writing with the writing implement. Alternatively, if the marking unit 11 is an inkjet printer, spray, or the like, the detection unit 15 can detect the start and end of marking by detecting the trigger operation of the inkjet printer, spray, or the like.

Figure 3 is a diagram showing an example of the external appearance of the tracking device 1 in this embodiment. In this example, the tracking device 1 is constructed by attaching a unit in which the position information generating unit 13 and the detecting unit 15 are implemented to the end or periphery of the chalk serving as the marking unit 11. The chalk and the unit are detachable, and the chalk is held inside the unit by frictional force. Pressure sensors and the like are disposed at the contact points between the chalk and the unit or the contact points between the user's hand and the unit, and the detecting unit 15 can detect the start and end of marking by analyzing the output of these sensors.

The location information generating unit 13 can start providing location information, i.e., acquiring and/or transmitting the location information, when the detection unit 15 detects the start of marking. The location information generating unit 13 can also end the acquisition and/or transmission of the location information when a predetermined time has elapsed since the start of the acquisition or transmission of the location information. More preferably, the location information generating unit 13 can end the acquisition and/or transmission of the location information when the detection unit 15 detects the end of marking. This allows the location information to be acquired and transmitted only while marking is being performed, making it possible to grasp the marking location more accurately. For example, the location information generating unit 13 can be configured to start acquiring location information when it detects the start of marking, accumulate the location information in a storage area (not shown), and transmit the accumulated location information when communication with the plotting device 2 becomes possible.

The plotting device 2 is a device that manages location information, and has a location information receiving unit 21, a location information storage unit 23, and a plotting unit 25. The plotting device 2 can typically be implemented in a PC (personal computer), a smartphone, a total station, etc.

The location information receiving unit 21 receives location information transmitted from the location information generating unit 13. The location information storage unit 23 stores the location information received by the location information receiving unit 21. The location information storage unit 23 can store the location information in association with information indicating the acquisition, transmission, or reception time or the acquisition, transmission, or reception order.

The plot unit 25 displays the location information stored in the location information storage unit 23. The location information may be plotted in a three-dimensional space, or may be displayed as two-dimensional information projected onto a specified plane. If time or sequence information is linked to the location information, a trajectory connecting points indicating the location information in chronological order may be plotted. The location information plotted by the plot unit 25 is basically expressed in a coordinate system with the plot device 2 as the origin, but if the location information generation unit 13 outputs location information based on another coordinate system, the location information may be plotted in the coordinate system used by the location information generation unit 13. Alternatively, if the coordinate system of the plot device 2 is expressed as a relative coordinate system with another base point (such as an AR marker or a total station) as the origin, the location information may be plotted in a coordinate system with the other base point as the origin.

According to this embodiment, the plotting system 100 electronically acquires and manages the location information of the location where a person has made a marking, thereby enabling the marking location to be grasped efficiently and accurately.

<Embodiment 2>
4 is a block diagram showing a configuration of a plotting system 100 according to the second embodiment. In the second embodiment, a tracking device 1 and a plotting device 2 are provided outside a writing implement or the like for marking.

Figure 5A is a diagram showing an example of the external appearance of the plotting system 100 in this embodiment. In this example, some or all of the functions of the tracking device 1 and the plotting device 2 are implemented within the housing of the upper limb wearable device. The upper limb wearable device is typically a small device such as a smartphone or a smartwatch, and is worn on the upper limb (e.g., wrist, arm, finger, etc.) of the user holding a writing implement or the like to perform marking.

The tracking device 1 has a location information generating unit 13 and a detecting unit 15. The plotting device 2 has a location information receiving unit 21, a location information storing unit 23, and a plotting unit 25.

The location information generating unit 13 acquires the location information of the tracking device 1 and transmits it to the plotting device 2.

For example, the position information generating unit 13 is equipped with a camera, and a base point such as an AR marker is set in advance within the field of view of the camera. The position information generating unit 13 can calculate its own coordinates with the AR marker as the origin by capturing an image including the AR marker with the camera and analyzing the angle, size, etc. of the AR marker. The position information generating unit 13 can also detect changes in its own position by autonomous navigation using an acceleration sensor, gyro sensor, magnetic sensor, etc.

The detection unit 15 can detect the start and end of marking, for example, by detecting changes in sound, speed, or acceleration generated when marking is performed with a writing implement or the like via various sensors. In this case, it is preferable that the upper limb wearable device as the plot system 100 is worn by the user on the arm holding the writing implement or the like. Alternatively, the detection unit 15 can detect the start and end of marking by detecting a user operation indicating the start or end of marking, for example, a specific tap operation, gesture, or voice.

Similarly to the first embodiment, the position information generating unit 13 can start acquiring and/or transmitting position information when the detecting unit 15 detects the start of marking.
Furthermore, the position information generating unit 13 can end the acquisition and/or transmission of the position information when a predetermined time has elapsed since the start of the acquisition or transmission of the position information.
Alternatively, the position information generating unit 13 can end the acquisition and/or transmission of the position information when the detecting unit 15 detects the end of the marking.

The position information generating unit 13 may store a preset offset (difference in coordinates) between the tracking device 1 and the writing implement, etc., and generate and transmit position information of the writing implement, etc., by adding the offset to the position information of the tracking device 1.

The position information receiving unit 21 of the plotting device 2 acquires the position information generated by the position information generating unit 13 at any time or at a predetermined timing, and accumulates it in the position information accumulating unit 23 in the same manner as in the first embodiment. The plotting unit 25 displays the position information accumulated in the position information accumulating unit 23 on the display of the head-mounted device.

As shown in FIG. 5B, some or all of the processing units of the plotting device 2 may be implemented in an external information processing device (such as a PC) rather than in the upper limb wearable device. For example, when only the plotting unit 25 is implemented in an external PC, the plotting unit 25 reads the position information stored in the position information storage unit 23 of the upper limb wearable device via wired or wireless communication means and performs plotting.

In this embodiment, the plotting system 100 can efficiently and accurately grasp the marking locations by electronically acquiring and managing the location information of the locations where people have made markings. In particular, the system can be easily implemented because it can be configured using only general-purpose devices such as smartphones.

<Third embodiment>
6 is a block diagram showing a configuration of a plotting system 100 according to the third embodiment. In the third embodiment, too, a tracking device 1 and a plotting device 2 are provided outside a writing implement or the like for marking.

FIG. 7A is a diagram showing an example of the appearance of the tracking device 1 in this embodiment. In this example, some or all of the functions of the tracking device 1 and the plot device 2 are implemented in the housing of the head-mounted device. The head-mounted device is typically an HMD (Head Mounted Display) having a camera, a depth sensor, and a transmissive display, and includes, for example, Hololens (registered trademark). The depth sensor can scan the real space to obtain three-dimensional point cloud data indicating the shape of an object at a short distance. The transmissive display allows the user to simultaneously view the real space and any information.
This can be realized, for example, by superimposing any desired information on an optically transparent or video transparent display. The head-mounted device is worn on the head of a user who holds a writing implement or the like and performs marking.

The tracking device 1 has a location information generating unit 13 and a detecting unit 15. The plotting device 2 has a location information receiving unit 21, a location information storing unit 23, and a plotting unit 25.

The position information generating unit 13 detects the position of the writing implement or other device used by the user to mark and transmits this to the plotting device 2.

For example, the position information generating unit 13 includes a camera and a depth sensor, and can continuously obtain image data with depth information. The position information generating unit 13 detects a writing implement or the like by analyzing the image data. This detection can be realized using various known techniques.
Furthermore, the position information generating unit 13 acquires three-dimensional point cloud data of the detected writing implement, etc., generates coordinates of representative points thereof, and sets them as the position of the writing implement, etc. The position of the writing implement, etc. is expressed as a relative position from an arbitrary origin in a coordinate system specific to the head-mounted device.

When a base point such as an AR marker is set in advance within the field of view of the camera, the position information generating unit 13 can calculate the coordinates of the position of the self (head-mounted device) with the AR marker as the origin by capturing an image including the AR marker with the camera and analyzing the angle, size, etc. of the AR marker. In this case, the position information generating unit 13 can convert the position information of a writing implement, etc. in a coordinate system specific to the head-mounted device into position information in a coordinate system with the AR marker as the origin.

The position information generating unit 13 can also detect changes in its own position through autonomous navigation using an acceleration sensor, gyro sensor, magnetic sensor, etc.

The detection unit 15 can detect the start and end of marking based on image data acquired by the camera. For example, the detection unit 15 can detect the start and end of marking by detecting a user's gesture or voice indicating the start or end of marking.

Similarly to the first embodiment, the position information generating unit 13 can start acquiring and/or transmitting position information when the detecting unit 15 detects the start of marking.
Furthermore, the position information generating unit 13 can end the acquisition and/or transmission of the position information when a predetermined time has elapsed since the start of the acquisition or transmission of the position information.
Alternatively, the position information generating unit 13 can end the acquisition and/or transmission of the position information when the detecting unit 15 detects the end of the marking.

The position information receiving unit 21 acquires the position information generated by the position information generating unit 13 at any time or at a predetermined timing, and accumulates it in the position information accumulating unit 23 in the same manner as in the first embodiment. The plotting unit 25 displays the position information accumulated in the position information accumulating unit 23 on the display of the head-mounted device.

As shown in FIG. 7B, some or all of the processing units of the plot device 2 may be implemented in an external information processing device (such as a PC) rather than in a head-mounted device. For example, when only the plot unit 25 is implemented in an external PC, the plot unit 25 reads the position information stored in the position information storage unit 23 of the head-mounted device via wired or wireless communication means and performs plotting.

In this embodiment, the plot system 100 can efficiently and accurately grasp the marking location by electronically acquiring and managing the position information of the location where a person has made a marking. In particular, the system can be easily introduced because it can be configured using only general-purpose devices such as an HMD with a depth sensor.

The present invention is not limited to the above-described embodiments, and can be modified as appropriate without departing from the spirit of the present invention. For example, the implementation method of the various processing means shown in the above-described embodiments using software or hardware is merely an example, and other methods may be used instead. For example, the multiple processing means implemented in the same housing in the embodiments can be provided as multiple different pieces of hardware. Or, the multiple processing means implemented in different pieces of hardware in the embodiments can be implemented in a single piece of hardware. In addition, any processing means or some of the functions of the processing means may be realized using technologies such as cloud computing, edge computing, fog computing, etc.

Furthermore, the communication and sensing methods described in the above embodiments are merely examples, and may be replaced by other methods having equivalent functions.

Furthermore, each processing means constituting the present invention may be configured by hardware, or any process may be realized by having a CPU execute a computer program. Furthermore, computer programs can be stored and supplied to a computer using various types of temporary or non-temporary computer-readable media. Temporary computer-readable media include, for example, electromagnetic signals supplied to a computer by wire or wirelessly.

REFERENCE SIGNS LIST 100 Plot system 1 Tracking device 11 Marking unit 13 Position information generating unit 15 Detection unit 2 Plot device 21 Position information receiving unit 23 Position information storage unit 25 Plot unit

Claims (9)

A tracking device worn on the body of a user who marks an object,
A detection unit that detects that the marking has started or ended;
a position information generating unit that generates position information of a tool for marking the object based on a position of the tracking device and an offset between the tracking device and the tool , and transmits the position information to a plotting device;
The position information generation unit starts at least one of generating and transmitting position information when the detection unit detects the start of the marking. A tracking device worn on the body of a user who marks an object,
A detection unit that detects that the marking has started or ended;
a position information generating unit that generates position information of a tool for marking the object based on the three-dimensional point cloud data of the tool and transmits the position information to a plotting device,
The position information generating unit starts at least one of generating and transmitting position information when the detecting unit detects the start of the marking.
Tracking devices. The detection unit detects the start or end of the marking by detecting a change in sensor output information including sound, speed, or acceleration, or an input of user information including a tap, gesture, or voice.
The tracking device of claim 1 . The position information generating unit ends at least one of the generation and transmission of the position information when the detecting unit detects the end of the marking.
A tracking device according to claim 1 or 2. The position information generating unit ends at least one of the generation and the transmission of the position information when a predetermined time has elapsed since starting the generation or the transmission of the position information.
A tracking device according to claim 1 or 2. The tracking device according to claim 1 or 2;
A plot device,
The plotting device comprises:
A location information receiving unit that receives the location information;
a location information storage unit that stores the received location information;
A plot unit that displays the accumulated position information.
Plot system. A tracking device is attached to the body of a user who marks an object,
A detection step of detecting that the marking has started or ended;
a position information generating step of generating position information of a tool for marking the object based on a position of the tracking device and an offset between the tracking device and the tool, and transmitting the position information to a plotting device;
In the position information generating step, at least one of generating and transmitting the position information is started when the start of the marking is detected.
Plotting method. A tracking device is attached to the body of a user who marks an object,
A detection step of detecting that the marking has started or ended;
a position information generating step of generating position information of a tool for marking the object based on the three-dimensional point cloud data of the tool and transmitting the position information to a plotting device,
In the position information generating step, at least one of generating and transmitting the position information is started when the start of the marking is detected.
Plotting method. A program for causing a computer to execute the method according to claim 7 or 8 .

Images