JP7377721B2 - Marker detection device and its program, and marker light emitting device - Google Patents

Description

The present invention relates to a marker detection device and its program, and a marker light emitting device.

A method for estimating the position, orientation, and angle of view (hereinafter referred to as camera parameters) of a camera is to use an object with a known three-dimensional shape and size, or multiple markers with a known three-dimensional positional relationship. Common. This estimation method is called strong calibration.
For example, camera parameters can be estimated by solving a Perspective 4-Point Problem (P4P problem) using four or more points with known positional relationships existing in the same plane.
In the camera calibration method using this method of solving the P4P problem, it is necessary to associate which image four or more points correspond to on the image captured by the camera.
In general, a plan-based marker on which a rectangular pattern of known shape and size is printed is used, and an identification pattern printed together with the rectangular pattern is used to distinguish a plurality of markers.
Furthermore, a method of estimating camera parameters using a pattern whose three-dimensional position is unknown and which is located near a point whose three-dimensional position is known has been disclosed (see Non-Patent Document 1).

Toshie Misu, Masato Fujii, Masahiro Shibata: “Camera parameter estimation method using SIFT features near calibration control points”, Proceedings of the 8th Information Science and Technology Forum (FIT2009), no. 3, H-049, pp. 207-212 (2009).

When a non-luminescent marker such as a printed pattern is used as a marker for strong calibration, the pattern cannot be used for calibration unless the marker is exposed to light.
Furthermore, even if a marker includes an identification pattern, if uniform illumination cannot be obtained, such as when a part of the identification pattern is cast in a shadow, an identification error may occur.
Furthermore, if an object with a pattern similar to the marker exists within the field of view of the camera to be calibrated, erroneous recognition will occur.
As described above, the conventional method has a problem in that marker detection and identification cannot be performed robustly. In addition to strong calibration, there is a need for a method for robustly detecting and identifying at least two or more markers, such as detecting the position of an object.

In view of these problems and demands, it is an object of the present invention to provide a marker detection device and its program, and a marker light emitting device that can robustly detect and identify markers.

In order to solve the above problem, a marker detection device according to the present invention provides a marker detection device according to the present invention, in which a light emission pattern vector indicating a change in a light emission pattern for each time slot is detected at the same marker in a time interval of a light emission pattern made up of a plurality of time slots. A marker detection device that detects a marker from a video image of a plurality of markers emitting light in a light emission pattern in which the inner product is greater than 0 and the inner product between different markers is less than or equal to 0, the marker detection device comprising: a correlation calculation unit; and a threshold processing unit. The configuration includes a section and a marker coordinate calculation section.

In such a configuration, the marker detection device uses the correlation calculation unit to calculate the correlation between the pixels of the frame image of the video and the light emission pattern vector for each light emission pattern for each time interval of the light emission pattern. generate a correlated image of
At this time, in the luminescence pattern of each marker, the inner product of luminescence pattern vectors for the same marker is greater than 0, and the inner product of luminescence pattern vectors between different markers is less than or equal to 0. Therefore, the correlation image is composed of pixels that have a strong correlation with the luminescence pattern. If the pixel value of is positive or negative, it becomes a large value.

Then, in the marker detection device, the threshold processing section binarizes the correlation image for each emission pattern using a predetermined threshold. Thereby, the marker detection device generates a binarized image in which the marker and other areas are separated.
Then, in the marker detection device, the marker coordinate calculation unit calculates a representative point such as the center of gravity of the marker region in the binarized image generated by the threshold processing unit.
Thereby, the marker detection device detects the position of the marker identified by the light emission pattern as the position of the representative point.

Note that the marker detection device can be operated by a marker detection program for causing a computer to function as each of the above-described units.

Further, in order to solve the above problems, a marker light emitting device according to the present invention is a marker light emitting device that emits a plurality of markers in a predetermined light emission pattern for each marker, and includes a plurality of light emitting elements serving as the plurality of markers. The present invention is configured to include a light emitting section having a light emitting section, and a light emitting control section controlling the light emitting state of each marker.

In such a configuration, in the marker light emitting device, the light emitting control unit causes the inner product of the light emitting pattern vectors indicating the change in the light emitting pattern for each time slot to be 0 at the same marker in the time interval of the light emitting pattern composed of a plurality of time slots. The light emitting state of the marker of the light emitting unit is controlled using a light emitting pattern that is larger and has an inner product between different markers of 0 or less.
By controlling the light emission state of the marker using such a light emission pattern, it becomes possible to detect a position on the image that has a strong correlation with the light emission pattern of the marker from an image of the marker.

The present invention has the following excellent effects.
According to the present invention, the marker can be detected even in an environment where illumination is not uniform due to the light emission of the marker. Further, according to the present invention, it is possible to robustly detect and identify a marker in an image based on the light emission pattern of the marker.

FIG. 1 is a block configuration diagram showing the configuration of a marker light emitting device and a marker detection device according to a first embodiment of the present invention. FIG. 3 is a diagram showing an example of marker arrangement. (a) to (f) are diagrams showing light emission patterns of different markers, respectively. (a) to (d) show examples of time-series frame images captured by the camera. (a) to (c) are diagrams showing examples of correlation images generated by a correlation calculation unit. FIG. 2 is a block configuration diagram showing the configuration of a threshold processing section. (a) to (f) are diagrams showing examples of binarized images generated by a threshold processing section. It is a flowchart showing the operation of the marker detection device according to the first embodiment of the present invention. FIG. 2 is a block configuration diagram showing the configuration of a marker detection device according to a second embodiment of the present invention. (a) to (d) show examples of time-series images captured by the camera when the posture changes. FIG. 3 is a block configuration diagram showing the configuration of a marker light emitting device according to a third embodiment of the present invention. FIG. 3 is a diagram showing an example of a marker made up of a plurality of light emitters. (a) to (f) are diagrams showing changes in the Walsh function over time. It is a block diagram showing the composition of a marker detection device concerning a 3rd embodiment of the present invention. FIG. 3 is a block configuration diagram showing the configuration of a marker light emitting device and a marker detection device according to a modified example.

Embodiments of the present invention will be described below with reference to the drawings.
<First embodiment>
[Marker detection system]
With reference to FIG. 1, an overview of a marker detection system S according to a first embodiment of the present invention will be described.
The marker detection system S detects and identifies two or more markers from an image captured by a camera.
The marker detection system S includes a marker light emitting device 1, a marker detection device 2, and a camera 3.

The marker light emitting device 1 emits light from a plurality of markers in a predetermined light emission pattern for each marker. Here, the marker light emitting device 1 and the marker detecting device 2 are connected by a signal line 4, and the light emitting pattern is outputted from the marker light emitting device 1 as a control signal to the marker detecting device 2 via the signal line 4.
The marker detection device 2 detects the position of the marker from the image captured by the camera 3 based on the light emission pattern of the marker emitted by the marker light emitting device 1. Here, the marker detection device 2 outputs the position of each marker as coordinates in the time-series images forming the video.

The camera 3 is a video camera that images the marker emitted by the marker light emitting device 1. Note that when the marker detection system S is used for camera calibration, the camera 3 becomes a camera to be calibrated.
The marker light emitting device 1 and the marker detection device 2 will be described in detail below.

[Marker light emitting device]
First, with reference to FIG. 1, a marker light emitting device 1 according to a first embodiment of the present invention will be described.
As shown in FIG. 1, the marker light-emitting device 1 includes a light-emitting section 10 and a light-emission control section 11.
The light emitting unit 10 includes a plurality of light emitting elements as a plurality of markers 101, . . . , 106.
The markers 101, . . . , 106 emit light (electromagnetic waves such as visible light, infrared light, and ultraviolet light) by switching between two light emission states. The two light emission states are a lighting state and a light-off state in the case of blinking, and a high brightness state and a low brightness state in the case of a change in brightness.
Here, an example will be explained in which there are six markers (101 to 106) and two light emission states are blinking.

Markers 101, . . . , 106, which are light emitting elements constituting the light emitting unit 10, are arranged in a space imaged by the camera 3, as shown in FIG.
The markers 101, .

The light emission control unit 11 controls the light emission state of each marker 101, . . . , 106. The light emission control section 11 controls the light emission state of the markers 101, . . . , 106 of the light emission section 10 according to a predetermined light emission pattern.
Here, the light emission control unit 11 uses a light emission pattern defined by a time function that takes binary values. Hereinafter, the time function is assumed to be w _n (t). Note that n is a code for distinguishing markers (here, 101 to 106), and t is a time. w _n (t) takes two values: "+1" and "-1".

The light emission control unit 11 controls the light emission state of each marker according to the value of the time function w _n (t) for each marker, such that, for example, the light is turned on when the value is "+1", and the light is turned off when the value is "-1". Control.
The time function w _n (t) has a pattern that is repeated in a predetermined time period, has a finite number of time slots (fixed time sections), and continues one light emitting state in each time slot. Let it be a function.
For example, the time function w _n (t) has M time slots synchronized with the frame rate F of imaging by the camera 3, and is defined by the following Walsh function W _n that repeats at a time period of M/F. (1).

また、演算子％は剰余（以下の数式においても同じ）であり、整数ａおよび非零の整数ｂに対して、以下の式（２）の演算を行うものである。

Further, the operator % is a remainder (the same applies to the following formulas), and is used to perform the calculation of the following formula (2) on an integer a and a non-zero integer b.

Furthermore, W _n () is a Walsh function excluding constant functions, and for example, when the number of time slots is M=4, W ₁₀₁ () to W ₁₀₆ () are expressed as the following equation (3). do.

The time function w _n (t) of the above equation (1) having the value of this equation (3) is shown in the graph of FIG.
In FIG. 3, (a) is w ₁₀₁ (t), (b) is w ₁₀₂ (t), (c) is w ₁₀₃ (t), (d) is w ₁₀₄ (t), and (e) is w ₁₀₅ (t) and (f) indicate the values of w ₁₀₆ (t) at time t for each of four time slots.
Note that each value of (d) to (f) is the value obtained by inverting the sign of the value of (a) to (c) at the same time.

Here, FIG. 4 shows the light emission state of the markers 101, . . . , 106 controlled by the above formula (1). FIG. 4 shows frame images that change over time in a 4-frame period of the video imaged by the camera 3, where (a) is I(4k+0), (b) is I(4k+1), and (c) is I(4k+2). ) and (d) show frame images of I(4k+3) (k is an integer).
FIG. 4A shows a state in which the markers 101, 102, and 103 are lit, and the markers 104, 105, and 106 are turned off. FIG. 4B shows a state in which the markers 101, 105, and 106 are lit, and the markers 102, 103, and 104 are turned off. FIG. 4C shows a state in which the markers 103, 104, and 105 are lit, and the markers 101, 102, and 106 are turned off. FIG. 4(d) shows a state in which the markers 102, 104, and 106 are lit, and the markers 101, 103, and 105 are turned off.
In this way, the light emission control unit 11 periodically changes the light emission state of the markers 101, . . . , 106 and causes them to emit light in a predetermined number of time slots.

The light emission pattern in which the light emission control unit 11 controls the _markers 101, . It is assumed that the inner product (cross-correlation) of the light emission pattern vectors in the time function w _m (t) (m≠n) is “0” or less.
The light emission pattern vector is a numerical value string indicating a change in the light emission pattern for each time slot. For example, w ₁₀₁ (t) in FIG. 3(a), where the number of time slots changes by four, is [ +1 +1 −1 −1 ], and w ₁₀₂ (t) in FIG. 3(b) is [ +1 −1 1 −1 +1 ], w ₁₀₃ (t) in FIG. 3(c) is [ +1 −1 +1 −1 ], and so on.

Note that the time function w _n (t) only needs to have a cross-correlation with the time function w _m (t) (m≠n) that is sufficiently lower than the autocorrelation of the time function w _n (t). is preferably 0.5 times or less than the autocorrelation.
Further, the time function w _n (t) may be any function other than the Walsh function, such as a pseudorandom noise sequence, an M sequence, or the like, which periodically switches between binary values (assumed to be ±1). In this case, the light emitting state of the marker may be associated with each of the periodically switched binary values.

In addition, although the light emitting part 10 was comprised here with six markers 101,...,106, the number of markers is not limited to this. When increasing or decreasing the number of markers, it is sufficient to cause each marker to emit light in an individual light emission pattern. In that case, the number of time slots may be increased or decreased, or signals with inverted signs (here, w ₁₀₄ (t) to w ₁₀₆ (t)) may not be used.

[Marker detection device: configuration]
Next, with reference to FIG. 1, the configuration of the marker detection device 2 according to the first embodiment of the present invention will be described.
As shown in FIG. 1, the marker detection device 2 includes a correlation calculation section 20, a threshold processing section 30, and a marker coordinate calculation section 40.

The correlation calculation unit 20 calculates the correlation between the pixels of the frame image of the video and the light emission pattern vector for each light emission pattern for each time interval of the light emission pattern. The result of this calculation is a correlation image made up of pixels with pixel values calculated as correlations for each light emission pattern.
Here, the correlation calculation section 20 includes a multiplication section 21 and an integration section 22.

For each light emission pattern, the multiplier 21 calculates the light emission of the light emission pattern input via the signal line 4 whose sign changes depending on the pixel value of the frame image of the video input from the camera 3 and the time of the frame image. This is to multiply the value of the pattern vector.
Here, the multiplier 21 converts the frame image I(t) input at time t and the time function w _n (t), which is a control signal for each marker, as shown in Equation (4) below. Multiplying by individual multipliers M ₁ to M ₃ generates a multiplication result p _n (t) for each marker.

Note that when the light emission control unit 11 of the marker light emitting device 1 uses an inverted light emission pattern as the light emission pattern, the multiplication unit 21 may use either a non-inverted light emission pattern or an inverted light emission pattern. .
Here, functions w ₁₀₁ (t), w ₁₀₂ (t), and w ₁₀₃ (t) indicating non-inverted light emission patterns are input to multipliers M ₁ , M ₂ , and M ₃ , respectively, and the frame image I (t) to generate multiplication results p ₁₀₁ (t), p ₁₀₂ (t), and p ₁₀₃ (t).
The multiplication section 21 outputs the multiplication results p ₁₀₁ (t), p ₁₀₂ (t), and p ₁₀₃ (t) to the integration section 22 .

The integrating section 22 integrates the multiplication results for each light emitting pattern multiplied by the multiplier 21 for each light emitting pattern time section. This integration result becomes a correlation image.
Here, the integration unit 22 multiplies the multiplication result p _n (t) generated by the multiplication unit 21 into M time slots (here, , 4) minutes, a correlation image r _n (t) is generated by calculating the sum for each pixel position.

Note that L is a natural number and is typically "1". This L means that the number of integrations in the integration unit 22 is not necessarily limited to the number of time slots, which is the reference for repeating the light emission pattern, but may be an integral multiple thereof.
The integration unit 22 outputs the generated correlation images r ₁₀₁ (t), r ₁₀₂ (t), and r ₁₀₃ (t) to the integration unit 22 .

Here, a specific example of the correlation image r _n (t) will be described with reference to FIG. 5.
FIG. 5 is a diagram showing correlation images generated by the integration unit 22 by the calculation of equation (5), in which (a) shows the correlation image r ₁₀₁ (t), and (b) shows the correlation image r ₁₀₂ (t). , (c) shows the correlated image t ₁₀₃ (t).
As shown in FIG. 5, the marker image of the marker 101 appears only in the correlation image r ₁₀₁ (t), and the marker image has a positive pixel value with high brightness according to the brightness of the marker 101, and other pixels The pixel value is lower in luminance than the pixel value.
Further, the marker image appears only in the correlation image r ₁₀₂ (t), and the marker image has a positive pixel value with high brightness according to the brightness of the marker 102, and other pixels have lower brightness. It becomes a pixel value.
Similarly, the marker image of the marker 103 appears only in the correlation image r ₁₀₃ (t), and the marker image has a positive pixel value with high brightness according to the brightness of the marker 103, and other pixels have a brightness higher than that. This results in a low pixel value.

Since the marker 104 is obtained by determining the sign of the light emission pattern of the marker 101, the marker image appears only in the correlation image r ₁₀₁ (t), and the marker image has a negative luminance with low luminance corresponding to the luminance of the marker 104. This is the pixel value, and other pixels have pixel values with higher luminance.
Furthermore, since the marker 105 is obtained by determining the sign of the light emission pattern of the marker 102, the marker image appears only in the correlation image r ₁₀₂ (t), and the marker image has a low brightness corresponding to the brightness of the marker 105. This becomes a negative pixel value, and other pixels take on pixel values with higher luminance.
Similarly, since the marker 106 is obtained by determining the sign of the light emission pattern of the marker 103, the marker image appears only in the correlation image r ₁₀₃ (t), and the marker image has a luminance corresponding to the luminance of the marker 106. It becomes a low negative pixel value, and other pixels have pixel values with higher luminance.
In this way, the correlation calculation unit 20 generates, as a correlation image, an image in which the marker area can be identified by brightness.
Returning to FIG. 1, the description of the configuration of the marker detection device 2 will be continued.

The threshold processing unit 30 binarizes the correlation image for each light emission pattern generated by the correlation calculation unit 20 using a predetermined threshold, thereby generating a binarized image in which markers and other areas are separated. It is something.
For a correlated image r _n (t) in which the image of the marker to be extracted has a positive pixel value, the threshold processing unit 30 performs a process in which the pixel value of each pixel of the correlated image r _n (t) exceeds a predetermined positive threshold. (or greater than or equal to a threshold value). Then, if the threshold value is exceeded (or is greater than or equal to the threshold value), the threshold processing unit 30 sets the pixel value of the corresponding pixel position of the binarized image to "1", and in other cases, the threshold value processing unit 30 sets the pixel value of the binarized image to For example, let the pixel value at the pixel position be "0".

In addition, for a correlated image r _n (t) in which the image of the marker to be extracted has a negative pixel value, the threshold processing unit 30 determines that the pixel value of each pixel of the correlated image r _n (t) has a predetermined negative threshold value. (or above a threshold). Then, the threshold processing unit 30 sets the pixel value at the corresponding pixel position in the binarized image to "0", for example, when the threshold value is exceeded (or the threshold value or more), and in other cases, the threshold processing unit 30 sets the pixel value at the corresponding pixel position in the binarized image to For example, the pixel value at the pixel position is "1".

For example, the threshold processing section 30 can have a configuration shown in FIG. 6.
As shown in FIG. 6, the threshold processing section 30 includes a plurality of binarization sections 31 (31 ₁ , 31 ₂ , 31 ₃ , 31 ₄ , 31 ₅ , 31 ₆ ) corresponding to the number of markers.
The binarization units 31 ₁ , 31 ₂ , 31 ₃ binarize the correlation images using positive thresholds.
Here, the binarization unit 31 ₁ inputs the correlation image r ₁₀₁ (t) and sets pixel values exceeding the threshold (+θ) to “1” and sets pixel values below the threshold (+θ) to “0”. Then, a binarized image b ₁₀₁ (t) is generated.
The binarization unit 31 ₂ generates a binarized image b ₁₀₂ (t) from the correlation image r ₁₀₂ (t) using the same process as the binarization unit 31 ₁ .
The binarization unit 31 ₃ also generates a binarized image b ₁₀₃ (t) from the correlation image r ₁₀₃ (t) using the same process as the binarization unit 31 ₁ .

The binarization units 31 ₄ , 31 ₅ , 31 ₆ binarize the correlation images using negative thresholds.
Here, the binarization unit ₃₁₄ inputs the correlation image r ₁₀₁ (t), and sets pixel values above the threshold (-θ) to "0" and pixel values below the threshold (-θ) to "1". By doing so, a binarized image b ₁₀₄ (t) is generated.
The binarization unit ₃₁₅ generates a binarized image b ₁₀₅ (t) from the correlation image r ₁₀₂ (t) using the same process as the binarization unit 31 ₄ .
The binarization unit ₃₁₆ also generates a binarized image b ₁₀₆ (t) from the correlation image r ₁₀₃ (t) using the same process as the binarization unit 31 ₄ .

For example, the threshold processing unit 30 generates the binarized images b ₁₀₁ (t) to b ₁₀₆ (t) shown in FIG. 7 from the correlation images r ₁₀₁ (t) to r ₁₀₃ (t) shown in FIG. do. Note that the image indicated by the dotted line in FIG. 7 shows the position of the marker in an easy-to-understand manner, and is an image that does not exist in the actual binarized image.
As shown in FIG. 7A, only the marker 101 is detected brightly in the binarized image b ₁₀₁ (t). Furthermore, as shown in FIG. 7B, only the marker 102 is detected brightly in the binarized image b ₁₀₂ (t). Furthermore, as shown in FIG. 7C, only the marker 103 is detected brightly in the binarized image b ₁₀₃ (t). Moreover, as shown in FIG. 7(d), only the marker 104 is detected brightly in the binarized image b ₁₀₄ (t). Further, as shown in FIG. 7E, only the marker 105 is detected brightly in the binarized image b ₁₀₅ (t). Moreover, as shown in FIG. 7(f), only the marker 106 is detected brightly in the binarized image b ₁₀₆ (t).
Returning to FIG. 1, the description of the configuration of the marker detection device 2 will be continued.
The threshold processing unit 30 outputs the generated binary image for each marker to the marker coordinate calculation unit 40.

The marker coordinate calculation unit 40 calculates representative points of the marker area in the binarized image generated by the threshold processing unit 30. The representative point is a position that is representative of an image area, and is the center of gravity, average, etc. of the area.
Here, the marker coordinate calculation unit 40 calculates, as marker coordinates, the center of gravity of an area of a predetermined pixel value (here, "1") detected as a marker image in each binarized image.
When the pixel value at the image coordinates (x, y) of the binarized image b _n (t) is b _n (t; x, y), the marker coordinate calculation unit 40 calculates the marker calculates the image coordinates (ξ _n , η _n ) of marker n as the marker coordinates.

The marker coordinate calculation unit 40 outputs the calculated marker coordinates for each marker as a detection result.
By configuring the marker detection device 2 as described above, the marker detection device 2 can detect markers and identify each marker even in an environment where illumination is not uniform based on the light emission pattern of the marker. .
Note that the marker detection device 2 can be operated by a program (marker detection program) that causes a computer to function as each section described above.

[Marker detection device: operation]
Next, with reference to FIG. 8 (see FIG. 1 as appropriate for the configuration), the operation of the marker detection device 2 according to the first embodiment of the present invention will be described. It is assumed that the marker detection device 2 receives an image captured by the camera 3 of a space in which the marker light emitting device 1 causes a plurality of markers to emit light in a predetermined light emission pattern. Further, it is assumed that a control signal for a light emitting pattern in which the marker light emitting device 1 causes the marker to emit light is input to the marker detection device 2 via the signal line 4.

In step S1, the multiplication unit 21 of the correlation calculation unit 20 generates a multiplication result for each emission pattern by multiplying each frame image of the video imaged by the camera 3 by the control signal of the emission pattern for each marker. Note that when the light emission patterns of the markers are inverted patterns, the multiplier 21 generates the multiplication result using only one of the light emission patterns.

In step S2, the integration unit 22 of the correlation calculation unit 20 calculates the sum of the multiplication results generated in step S1 for each pixel position in a time interval that serves as a reference for repeating the light emission pattern.
Thereby, the correlation calculation unit 20 generates, for each light emission pattern, a correlation image (see FIG. 5) that shows the correlation for each pixel based on the time interval of the light emission pattern.

In step S3, the threshold processing unit 30 generates a binarized image for each marker (see FIG. 7) by performing threshold processing on the correlation image for each emission pattern generated in step S2.
Note that when the light emission patterns of the markers are inverted patterns, the threshold processing unit 30 generates two binarized images from one correlation image.

In step S4, the marker coordinate calculation unit 40 calculates the coordinates of a representative point (centroid) of a region of a predetermined pixel value indicating the region of the marker from the binarized image for each marker generated in step S3.
Thereby, the marker coordinate calculation unit 40 calculates the marker coordinates on the image for each marker.
Through the above-described operations, the marker detection device 2 is able to detect and identify markers from images of the markers based on the light emission patterns of the plurality of markers.

<Second embodiment>
[Marker detection device]
Next, with reference to FIG. 9, a marker detection device 2B according to a second embodiment of the present invention will be described. Note that the marker light emitting device 1 described in FIG. 1 can be used as the marker light emitting device.

Like the marker detection device 2, the marker detection device 2B detects and identifies the position of the marker from the image captured by the camera 3 based on the light emission pattern of the marker emitted by the marker light emitting device 1. However, the marker detection device 2 requires that the position, attitude, and angle of view of the camera 3 be constant at least in the time interval of the light emission pattern. On the other hand, the marker detection device 2B detects markers by allowing movement of the position, posture, angle of view, etc. of the camera 3.

As shown in FIG. 9, the marker detection device 2B includes a correlation calculation section 20B, a threshold processing section 30, and a marker coordinate calculation section 40. The threshold processing unit 30 and the marker coordinate calculation unit 40 have the same configuration as the marker detection device 2 described in FIG.

The correlation calculation unit 20B calculates the correlation between the pixels of the frame image of the video and the light emission pattern vector for each light emission pattern, for each time interval of the light emission pattern, while allowing for changes in the position, orientation, and angle of view of the camera 3. By calculating , a correlation image is generated.
Here, the correlation calculation section 20B includes a video delay section 23, a motion compensation section 24, a signal delay section 25, a multiplication section 21B, and an integration section 22B.

The video delay unit 23 delays the video imaged by the camera 3 in units of predetermined frames. Here, a plurality of video delay units 23 ₁ , 23 ₂ , and 23 ₃ are provided, and each video delay unit 23 delays the video by one frame.
The video delay unit 23 ₁ delays the video input from the camera 3 by one frame and outputs the delayed video to the video delay unit 23 ₂ and the motion compensation unit 24 ₁ .
The video delay unit 23 ₂ delays the video input from the video delay unit 23 ₁ by one frame, and outputs the delayed video to the video delay unit 23 ₃ and the motion compensation unit 24 ₂ .
The video delay unit 23 ₃ delays the video input from the video delay unit 23 ₂ by one frame, and outputs the delayed video to the motion compensation unit 24 ₃ .

The motion compensation unit 24 estimates the motion of the video between the video input from the camera 3 and the video delayed by the video delay unit 23, and based on the estimated amount of motion, the motion of the video input from the camera 3 and the video delayed by the video delay unit 23. This method performs motion compensation for each frame image. Here, a plurality of motion compensation units 24 ₁ , 24 ₂ , and 24 ₃ are provided, and each motion compensation unit 24 performs motion compensation for the video delayed by the video delay units 23 ₁ , 23 ₂ , and 23 ₃ , respectively.

The motion compensation unit 24 ₁ estimates the motion of the image between the video input from the camera 3 and the video delayed by the video delay unit 23 ₁ and performs motion compensation, so that the delayed video is transmitted to the camera 3. A motion compensated video is generated that is aligned with the video input from. The motion compensation unit ₂₄₁ outputs the generated motion compensated video to the multiplication unit ₂₁₂ .
The motion compensation unit 24 ₂ estimates the motion of the image between the video input from the camera 3 and the video delayed by the video delay unit 23 ₂ and performs motion compensation, thereby transmitting the delayed video to the camera 3. A motion compensated video is generated that is aligned with the video input from. The motion compensation unit ₂₄₂ outputs the generated motion compensated video to the multiplication unit ₂₁₃ .
The motion compensation unit 24 ₃ estimates the motion of the image between the video input from the camera 3 and the video delayed by the video delay unit 23 ₃ and performs motion compensation, so that the delayed video is transferred to the camera 3. A motion compensated video is generated that is aligned with the video input from. The motion compensation unit ₂₄₃ outputs the generated motion compensated video to the multiplication unit ₂₁₄ .

Motion compensation in each motion compensation unit 24 may be performed using a general method.
For example, the motion compensation unit 24 may perform global motion compensation that geometrically transforms the entire screen. By performing global motion compensation, it is possible to compensate for the global motion of the entire screen and to detect and identify robust markers that are not affected by minute motions between images taken at different times.

Further, for example, the motion compensation unit 24 may perform local motion compensation by geometrically transforming the screen for each partial region. These partial areas may not overlap with each other or may overlap with each other. Furthermore, local motion compensation divides an image into a plurality of rectangular regions and applies individual geometric transformations to each rectangular region. By performing local motion compensation, highly accurate motion compensation is possible even when there is geometric distortion in the optical system of the camera 3, and the accuracy of marker detection and identification is improved.
Note that the geometric transformation used for global motion compensation and local motion compensation includes one or more combinations of translation, expansion/reduction, rotation, shear mapping (shear), affine transformation, homography transformation, and lens distortion compensation. Can be mentioned.

Further, motion estimation in each motion compensator 24 may be performed using a general method.
For example, the motion compensation unit 24 may estimate motion between images of the input video so as to minimize errors in pixel value sequences, or may maximize the correlation of pixel value sequences between images. It may also be possible to estimate the movement as follows. Moreover, a robust estimation method may be used as a method for minimizing errors in a pixel value sequence.
For example, when using the least median of squares (LMedS) as the robust estimation method, the motion compensation unit 24 uses Motion is estimated and motion compensation is performed by determining the error and calculating the geometric transformation parameters so that the median value becomes the minimum.
As a result, the robustness of motion compensation can be increased by suppressing the effects of changes in brightness and movement of subjects other than markers, image noise, etc., and robust detection and identification of markers becomes possible.

For example, as shown in FIGS. 10(a) to 10(d), a frame image I(t) at time t, a frame image I(t-1) at time t-1, a frame image I(t) at time t-2, -2), frame image I(t-3) at time t-3.
At this time, the motion compensation unit ₂₄₁ performs motion compensation on the frame image I(t-1) so that the error with the frame image I(t) is minimized. Further, the motion compensation unit ₂₄₂ performs motion compensation on the frame image I(t-2) so that the error between it and the frame image I(t) is minimized. Furthermore, the motion compensation unit ₂₄₃ performs motion compensation on frame image I(t-3) so that the error between it and frame image I(t) is minimized. Further, the motion compensator ₂₄₄ performs motion compensation on frame image I(t-4) so that the error between frame image I(t) and frame image I(t) is minimized.

The signal delay unit 25 delays the light emission pattern control signal input via the signal line 4 by a predetermined frame time unit. Here, a plurality of signal delay sections 25 ₁ , 25 ₂ , and 25 ₃ are provided, and each signal delay section 25 delays the control signal by a time corresponding to one frame. The amount of delay in each signal delay section 25 is the same as the amount of delay in each video delay section 23.

The signal delay section ₂₅₁ delays the control signal input via the signal line 4 by a time corresponding to one frame, and outputs the delayed control signal to the multiplication section ₂₁₂ and the signal delay section ₂₅₂ .
The signal delay section ₂₅₂ further delays the control signal delayed by the signal delay section ₂₅₁ by a time corresponding to one frame, and outputs the delayed control signal to the multiplication section ₂₁₃ and the signal delay section ₂₅₃ .
The signal delay section ₂₅₃ further delays the control signal delayed by the signal delay section ₂₅₂ by a time corresponding to one frame, and outputs the delayed control signal to the multiplication section ₂₁₄ .

The multiplier 21B multiplies each frame image of the video input from the camera 3 by a light emission pattern control signal input via the signal line 4. Here, a plurality of multipliers 21 ₁ , 21 ₂ , 21 ₃ , and 21 ₄ are provided, and each multiplier 21 sequentially multiplies the delayed video by the delayed control signal. Note that the multipliers 21 ₁ , 21 ₂ , 21 ₃ , and 21 ₄ each have the same configuration as the multiplier 21 described in FIG. 1 .

The multiplier 21 ₁ multiplies the frame image I(t) input at time t by the time function w _n (t), which is a control signal, using individual multipliers M ₁₁ to M ₁₃ .
The multiplier _M11 multiplies the frame image I(t) by the function _w101 (t), and outputs the multiplication result (○A in the figure) to the integration unit 22B. Note that the characters A, B, . . . , etc. enclosed in circles in the drawings are written as 〇A, 〇B, . . . in this specification.
The multiplier _M12 multiplies the frame image I(t) by the function _w102 (t), and outputs the multiplication result (○B in the figure) to the integration unit 22B.
The multiplier _M13 multiplies the frame image I(t) by the function _w103 (t), and outputs the multiplication result (○C in the figure) to the integration unit 22B.

The multiplier ₂₁₂ multiplies the frame image I(t-1) input at time t-1 and the time function w _n (t-1), which is a control signal, using individual multipliers M ₂₁ to M ₂₃ . Multiply.
The multiplier M ₂₁ multiplies the frame image I(t-1) by the function w ₁₀₁ (t-1) and outputs the multiplication result (○D in the figure) to the integration unit 22B.
The multiplier M ₂₂ multiplies the frame image I(t-1) by the function w ₁₀₂ (t-1), and outputs the multiplication result (○E in the figure) to the integration unit 22B.
The multiplier M ₂₃ multiplies the frame image I(t-1) by the function w ₁₀₃ (t-1), and outputs the multiplication result (○F in the figure) to the integration unit 22B.

The multiplier ₂₁₃ multiplies the frame image I(t-2) input at time t-2 and the time function w _n (t-2), which is a control signal, using individual multipliers M ₃₁ to M ₃₃ . Multiply.
The multiplier M ₃₁ multiplies the frame image I(t-2) by the function w ₁₀₁ (t-2), and outputs the multiplication result (○G in the figure) to the integration unit 22B.
The multiplier M ₃₂ multiplies the frame image I(t-2) by the function w ₁₀₂ (t-2) and outputs the multiplication result (○H in the figure) to the integration unit 22B.
The multiplier M ₃₃ multiplies the frame image I(t-2) by the function w ₁₀₃ (t-2), and outputs the multiplication result (○I in the figure) to the integration unit 22B.

The multiplier ₂₁₄ multiplies the frame image I(t-3) input at time t-3 and the time function w _n (t-3), which is a control signal, using individual multipliers M ₄₁ to M ₄₃ . Multiply.
Multiplier M ₄₁ multiplies frame image I(t-3) by function w ₁₀₁ (t-3) and outputs the multiplication result (〇J in the figure) to integration unit 22B.
The multiplier M ₄₂ multiplies the frame image I(t-3) by the function w ₁₀₂ (t-3), and outputs the multiplication result (○K in the figure) to the integration unit 22B.
The multiplier M ₄₃ multiplies the frame image I(t-3) by the function w ₁₀₃ (t-3) and outputs the multiplication result (○L in the figure) to the integration unit 22B.

The integrating unit 22B calculates the sum of the multiplication results generated by the multiplying unit 21B for each pixel position in a predetermined time interval. This integrating unit 22B calculates the sum of the multiplication results obtained by multiplying the same marker by the corresponding control signal. Here, the integrating section 22B includes an adding section 22 ₁ , an adding section 22 ₂ , and an adding section 22 ₃ .

The adding unit 22 ₁ combines the multiplication result (○A) in the multiplier M ₁₁ of the multiplication unit 21 ₁ , the multiplication result (○D) in the multiplier M ₂₁ of the multiplication unit 21 ₂ , and the multiplication result (○D) in the multiplier M 11 of the multiplication unit 21 ₃ . ₃₁ and the multiplication result (0J) in the multiplier _M41 of the multiplication unit ₂₁₄ are added.
The addition section ₂₂₁ outputs the addition result to the threshold processing section 30.

The adding unit ₂₂₂ combines the multiplication result (〇B) in the multiplier M12 of the multiplication unit ₂₁₁ , the multiplication result (〇E) in the multiplier _M22 of the multiplication unit 212, and the multiplication result (〇E) in the multiplier M12 of the multiplication unit 212, and the multiplication result (〇E) in the multiplier _M12 of the multiplication unit ₂₁₂ , and the multiplication result (〇E) in the multiplier _M12 of the multiplication unit 212. The multiplication result (○H) in ₃₂ and the multiplication result (○K) in multiplier M ₄₂ of the multiplication unit ₂₁₄ are added.
The addition section ₂₂₂ outputs the addition result to the threshold processing section 30.

The adding unit ₂₂₃ combines the multiplication result (○C) in the multiplier _M13 of the multiplication unit ₂₁₁ , the multiplication result (○F) in the multiplier _M23 of the multiplication unit ₂₁₂ , and the multiplication result (○F) in the multiplier _M13 of the multiplication unit 213. ₃₃ and the multiplication result (○L) in the multiplier M ₄₃ of the multiplication unit ₂₁₄ are added.
The addition section ₂₂₃ outputs the addition result to the threshold processing section 30.
Thereby, the integration unit 22B can generate a correlation image showing the correlation for each pixel with respect to the light emission pattern as shown in FIG.

By configuring the marker detection device 2B as described above, the marker detection device 2B can detect markers and identify each marker even in an environment where illumination is not uniform based on the light emission pattern of the marker. . Furthermore, the marker detection device 2B can accurately detect and identify markers even if the position, posture, and angle of view of the camera 3 are unstable.
Note that the marker detection device 2B can be operated by a program (marker detection program) that causes a computer to function as each section described above.

<Third embodiment>
[Marker light emitting device]
Next, with reference to FIG. 11, a marker light emitting device 1C according to a third embodiment of the present invention will be described.
The marker light emitting device 1 (see FIG. 1) of the first embodiment and the second embodiment controls the two light emitting states of the markers 101 to 106, lighting and turning off, using a predetermined light emission pattern.
In addition to turning on and off, this marker may form a light emitting pattern in a plurality of light emitting states with different brightness, different colors, or a combination thereof.
Here, a marker that emits light in a time slot that takes four states by turning off and emitting light in three different colors will be described as an example.
The marker light-emitting device 1C emits light-emitting patterns for detecting markers by turning off and emitting light in three different colors.
As shown in FIG. 11, the marker light emitting device 1C includes a light emitting section 10C and a light emission control section 11C.

The light emitting unit 10C includes a plurality of (here, 36) markers 101C, . . . , 136C.
The markers 101C, . . . , 136C emit light by switching between four light emitting states: off, red lighting, green lighting, and yellow lighting.

For example, as shown in FIG. 12, the marker 101C is a light emitting element in which red (R) and green (G) light emitters are enclosed in one exterior.
When both the red and green light emitters are turned off, the marker 101C is turned off. Further, when only the red light emitter is lit, the marker 101C is in a red lit state. Furthermore, when only the green light emitter is lit, the marker 101C is in a green lit state. Further, when both the red and green light emitters are turned on, the marker 101C is in a yellow lighting state.
The markers 101C, . . . , 136C switch their light emission states according to control signals output from the light emission control section 11C.

The light emission control section 11C controls the light emission state of the markers 101C, . . . , 136C of the light emission section 10C according to a predetermined light emission pattern.
Here, the light emission control unit 11C generates a light emission pattern using a time function that takes a binary value. Hereinafter, the time functions are assumed to be w _p ^(R) (t) and w _p ^(G) (t). Note that w _p ^(R) (t) is a function that controls the red (R) light emitter of the marker, and w _p ^(G) (t) is a function that controls the green (G) light emitter of the marker. be. Furthermore, p is a code (here, 0 to 35) that distinguishes markers, and t is a time. w _p ^(R) (t) and w _p ^(G) (t) each take two values: “+1 (light emission)” and “−1 (light off)”.
The time functions w _p ^(R) (t) and w _p ^(G) (t) are defined by the following equation (7).

The functions on the right side of equation (7) are six time functions w _q (t), (qε{0, 1, . . . , 5)), and are, for example, the Walsh functions shown in FIG. 13. In this case, a total of 36 states can be expressed by generating six light emission patterns for each of the red and green light emission of the marker.

The light emission control unit 11C controls the light emission unit 10C based on the time function shown in equation (7), and also uses a light emission pattern based on the time function as a control signal to send a marker detection device 2C (described later) via the signal line 4. Output to. Note that the signal line 4 ₁ is a signal line that transmits a signal indicating a light emission pattern based on the time function w _p ^(R) (t), and the signal line 4 ₂ is a signal line that transmits a signal indicating a light emission pattern based on the time function w _p ^(G) (t). This is a signal line that transmits a signal indicating a light emission pattern based on.
As a result, the marker light emitting device 1C can emit marker light in 36 light emitting patterns depending on whether the light is off or the light emitting state is in three different colors.

[Marker detection device]
Next, with reference to FIG. 14, a marker detection device 2C according to a third embodiment of the present invention will be described.
The marker detection device 2C detects and identifies the position of the marker from the color image captured by the camera 3C based on the light emission pattern of the marker emitted by the marker light emitting device 1C described in FIG. 11.

As shown in FIG. 14, the marker detection device 2C includes a correlation calculation unit 20 (20 ₁ , 20 ₂ ), a threshold processing unit 30 (30 ₁ , 30 ₂ ), a logical calculation unit 50, and a marker coordinate calculation unit 40. and.

The correlation calculation unit 20 generates a correlation image by calculating the correlation between the pixels of the frame image of the video and the light emission pattern vector for each light emission pattern for each time interval of the light emission pattern.
The correlation calculation section 20 includes two correlation calculation sections 20 ₁ and 20 ₂ corresponding to two color channels among the color channels imaged by the camera 3 . Note that the correlation calculation units 20 ₁ and 20 ₂ are the same as the correlation calculation unit 20 described in FIG. 1, except that the color channels of the input frame images are different. Note that the correlation calculation units 20 ₁ and 20 ₂ may be configured by the correlation calculation unit 20B described in FIG. 9 .

The correlation calculation unit ₂₀₁ receives the red channel video of the video captured by the camera 3C and the control signal generated by the marker light emitting device 1C corresponding to the red channel, and generates a correlation image. .
Here, the correlation calculation unit ₂₀₁ generates three correlation images for each time period corresponding to four time slots, which are the reference for repeating the light emission pattern (see FIG. 5). The correlation calculation unit ₂₀₁ outputs the generated correlation image to the threshold processing unit ₃₀₁ .

The correlation calculation unit ₂₀₂ receives the green channel video of the video captured by the camera 3C and the control signal generated by the marker light emitting device 1C corresponding to the green channel, and generates a correlation image. .
Here, the correlation calculation unit ₂₀₂ generates three correlation images for each time interval corresponding to four time slots, which are the reference for repeating the light emission pattern (see FIG. 5). The correlation calculation unit ₂₀₂ outputs the generated correlation image to the threshold processing unit ₃₀₂ .

The threshold processing unit 30 binarizes the correlation image for each light emission pattern generated by the correlation calculation unit 20 using a predetermined threshold, thereby generating a binarized image in which markers and other areas are separated. It is something.
The threshold processing section 30 includes two threshold processing sections 30 ₁ and 30 ₂ corresponding to the correlation calculation sections 20 ₁ and 20 ₂ . Note that the threshold processing units 30 ₁ and 30 ₂ are the same as the threshold processing unit 30 described in FIG. 1 .

The threshold processing unit ₃₀₁ generates a binarized image by performing threshold processing on the correlation image generated from the red channel video by the correlation calculation unit ₂₀₁ .
Here, the threshold processing unit ₃₀₁ generates six binarized images (see FIG. 6). The threshold processing section ₃₀₁ outputs the generated binarized image to the logical operation section 50.

The threshold processing unit ₃₀₂ generates a binarized image by performing threshold processing on the correlation image generated from the green channel video by the correlation calculation unit ₂₀₂ .
Here, the threshold processing unit ₃₀₂ generates six binarized images (see FIG. 6). The threshold processing section ₃₀₂ outputs the generated binarized image to the logic operation section 50.

The logical operation unit 50 calculates a logical product in the combination of the plurality of binarized images generated by the threshold processing unit ₃₀₁ and the plurality of binarized images generated by the threshold processing unit ₃₀₂ . be.
Here, the logical operation unit 50 generates 36 combinations of 6 binarized images generated by the threshold processing unit 30 ₁ and 6 binarized images generated by the threshold processing unit 30 ₂ . , calculates the logical product.
Note that the logical product in the logical operation unit 50 may be performed by selecting the minimum value of pixel values for each corresponding pixel position of the binarized images.
Thereby, the logic operation unit 50 can generate, for each marker, an image in which only the image of the marker is extracted.
The logic operation section 50 outputs the generated binarized images (here, 36 binarized images) to the marker coordinate operation section 40.

The marker coordinate calculation unit 40 calculates the coordinates of a representative point on the marker image as marker coordinates from the binary image for each marker generated by the logic calculation unit 50.
Note that the marker coordinate calculation section 40 is the same as the marker coordinate calculation section 40 described with reference to FIG. 1, except that the number of binarized images is different.
As a result, the marker detection device 2C has four time slots, four light-emitting states (including a light-off state), and six types of time functions w _q (t) (q∈0, 1, ..., 5) (Fig. 13), the positions of up to 36 markers can be detected and identified.
Note that the marker detection device 2C can be operated by a program (marker detection program) that causes a computer to function as each section described above.

Although the embodiments of the present invention have been described above, the present invention is not limited to these embodiments.
For example, here, there are two light-emitting states of the marker: on and off.
However, the light emitting state may be multi-state depending on the luminance value of the marker. Further, the light emitting state of the marker may be set to multiple states depending on the color (red, green, blue, cyan, magenta, yellow, white, etc.) and brightness value.
In that case, the marker detection device 2 generates images for each specific luminance range from the frame images captured by the camera 3, or generates images for each specific luminance range for each color channel of the frame images captured by the camera 3. It is sufficient to include a luminance-specific image generation unit (not shown) that generates a luminance-specific image generation unit (not shown), and output images for each luminance range to the plurality of correlation calculation units 20 of the marker detection device 2C described in FIG. 14.
Furthermore, a plurality of correlation calculation units 20 and threshold processing units 30 may be provided for each color channel and luminance range.
This makes it possible to place multivalued information in the time slot of the time interval that is the reference of the light emission pattern, and it becomes possible to detect more markers in a short time.

Further, here, the marker light emitting device 1 is configured to output the light emitting pattern as a control signal to the marker detection device 2 via the signal line 4.
However, as shown in FIG. 15, the marker detection device 2 includes a light emission control section 60 that generates the same light emission pattern as the light emission control section 11 of the marker light emitting device 1, and the light emission control section 11 and the light emission control section 60 It may also be configured to operate synchronously.
In this synchronization method, a signal for synchronization may be transmitted from one of the light emission control section 11 and the light emission control section 60 to the other by wire or wirelessly, or a signal source for synchronization provided externally may be used. The synchronization signal may be given to the light emission control section 11 and the light emission control section 60 by wire or wirelessly. Alternatively, the light emission control section 11 and the light emission control section 60 may be synchronized based on standard radio waves, beacons, positioning satellites, electromagnetic waves from celestial bodies, etc.

1 Marker light emitting device 10,0C Light emitting section 101,...,106 Marker 11,11C Light emission control section 2,2B,2C Marker detection device 20,20B Correlation calculation section 21,22B Multiplying section 22,22B Integrating section 23 Video delay section 24 Motion compensation unit 25 Signal delay unit 30 Threshold processing unit 31 Binarization unit 40 Marker coordinate calculation unit 50 Logical calculation unit 60 Light emission control unit 3, 3C camera

Claims (10)

In a time interval of a light emission pattern composed of a plurality of time slots, an inner product of light emission pattern vectors at the same marker indicating a change in the light emission pattern for each time slot is greater than 0, and the inner product between different markers is less than or equal to 0. A marker detection device that detects a plurality of markers from a captured image of a plurality of markers that emit light in a light emission pattern,
a correlation calculation unit that generates a correlation image for each of the light emission patterns by calculating a correlation between pixels of a frame image of the video and a light emission pattern vector for each of the light emission patterns for each of the time intervals;
a threshold processing unit that generates a binarized image in which the marker and other areas are separated by binarizing the correlation image for each of the light emission patterns using a predetermined threshold;
a marker coordinate calculation unit that calculates a representative point of the marker area in the binarized image generated by the threshold processing unit;
A marker detection device comprising: The correlation calculation unit is
a multiplication unit that multiplies, for each of the light emission patterns, a pixel value of a frame image of the video by a value of the light emission pattern vector whose sign changes depending on the time of the frame image;
an integrating unit that generates the correlation image by integrating the multiplication results for each of the light emission patterns multiplied by the multiplier for each of the time intervals;
The marker detection device according to claim 1, comprising: The correlation calculation unit is
a video delay unit that delays the video in units of frames;
a signal delay unit that delays a control signal indicating the light emission pattern in units of the frame in accordance with the time of the frame image;
a motion compensation unit that estimates motion of a video from a current frame image and the delayed frame image, and performs motion compensation for the delayed frame image;
a multiplication unit that multiplies, in the motion-compensated frame image, a pixel value of a pixel of the frame image by a value of the control signal delayed in accordance with a time of the frame image;
an integrating unit that generates the correlation image by adding multiplication results corresponding to the same light emission pattern multiplied by the multiplying unit for each of the time intervals;
The marker detection device according to claim 1, comprising: 4. The marker detection device according to claim 3, wherein the motion compensation unit performs global motion compensation or motion compensation for each partial region of the frame image. A plurality of the correlation calculation units and the threshold processing unit generate the correlation image for each light emission pattern indicating a plurality of light emission states with different brightness, different colors, or a combination thereof, and generate the binarized image. Prepare,
further comprising a logical operation unit that generates a binary image indicating an area of the marker related to a combination of light emission patterns by performing a logical product operation on the binary images generated by the plurality of threshold processing units; The marker detection device according to any one of claims 1 to 4, further comprising: a marker detection device according to claim 4. A marker detection program for causing a computer to function as the marker detection device according to any one of claims 1 to 5. A marker light emitting device that emits light from a plurality of markers in a predetermined light emission pattern for each marker,
a light emitting unit including a plurality of light emitting elements serving as the plurality of markers;
and a light emission control unit that controls the light emission state of each marker,
The light emission control unit is arranged such that, in a time interval of a light emission pattern made up of a plurality of time slots, an inner product of light emission pattern vectors at the same marker indicating a change in the light emission pattern for each time slot is greater than 0, and an inner product between different markers is larger than 0. A marker light emitting device, characterized in that the light emitting state of the marker is controlled by a light emitting pattern in which the inner product is 0 or less. 8. The marker light emitting device according to claim 7, wherein the light emitting state is two or more states of different brightness, different color, or a combination thereof. The marker light emitting device according to claim 7 or 8, wherein the light emission pattern is a pattern whose sign changes in the time interval based on a Walsh function, a pseudorandom noise sequence, or an M sequence. . 10. The marker light-emitting device according to claim 7, wherein one or more of the light-emitting patterns are patterns in which signs of other light-emitting patterns are reversed.

Images