JP7742484B2 - Detection system for detecting boarding and disembarking areas of an aircraft - Google Patents

Description

The present disclosure relates to a detection system for detecting boarding and disembarking areas of an aircraft.

Passenger boarding bridges are known as facilities that serve as pedestrian walkways for passengers between airport terminal buildings and aircraft. Passenger boarding bridges comprise a rotunda connected to the terminal building and supported for horizontal rotation; a tunnel section whose base end is connected to the rotunda and configured for telescopic movement; a cab rotatably mounted at the end of the tunnel section and attached to the aircraft boarding and disembarking section; and a drive column mounted as a support leg near the end of the tunnel section. The drive column comprises an elevator device that supports and moves the tunnel section up and down, and a traveling device with a pair of traveling wheels mounted below the elevator device. Automating the movement of such passenger boarding bridges has been proposed (see, for example, Patent Documents 1 and 2 listed below).

For example, Patent Document 1 describes a method in which a camera that photographs the boarding and disembarking area of an aircraft is attached to the cab, and when the cab is in a designated waiting position, horizontal position information of the boarding and disembarking area is calculated based on the image of the area photographed by the camera, and a target position to which the cab should be moved in order to attach it to the boarding and disembarking area is calculated based on this horizontal position information, and the cab, which is in the waiting position, is moved toward the target position.

Patent Document 2 also describes a configuration in which a head unit (cab) connectable to an aircraft boarding/alighting door is equipped with two cameras, a first and a second. When a drive start input is made on the control panel, the travel drive unit starts wheel travel, and when the head unit reaches a few meters in front of the aircraft, the first and second cameras begin capturing images of a first characteristic feature and a second characteristic feature of the aircraft. The images captured by the first and second cameras are then used to calculate the position of the target point of the aircraft boarding/alighting door, and the relative position and relative angle of the head unit with respect to the aircraft boarding/alighting door are calculated. Based on these, a control correction amount is calculated, and various drive units are driven based on this to move the head unit toward the target point of the aircraft.

Patent No. 6720414 Japanese Patent Application Laid-Open No. 2020-175727

To automate the movement of such passenger boarding bridges, it is necessary to accurately detect the position of the aircraft's boarding and disembarking areas from images captured by a camera installed at the tip of the passenger boarding bridge. In particular, the position of an aircraft's boarding and disembarking areas can vary greatly depending on the type and size of the parked aircraft. Furthermore, airport gate layouts vary widely, making it cumbersome to pre-set the boarding and disembarking area position detection method for each combination of parked aircraft and gate layout. Thus, there is room for improvement in the above-mentioned Patent Documents 1 and 2 in terms of accurately detecting the positions of boarding and disembarking areas for multiple types of aircraft in response to various gate layouts.

This disclosure has been made to solve the above-mentioned problems, and aims to provide a detection system that can accurately detect the position of an aircraft's boarding and disembarking area in accordance with various gate configurations using images captured by a camera installed at the tip of a passenger boarding bridge.

A detection system according to one aspect of the present disclosure is a detection system for detecting the boarding and disembarking area of an aircraft, comprising: a passenger boarding bridge connected to a terminal building; a first camera and a second camera provided on the passenger boarding bridge; a height change mechanism capable of changing the height positions of the first camera and the second camera; a controller for controlling the operation of the height change mechanism; and an image processing device for detecting the boarding and disembarking area of the aircraft from a first image taken by the first camera and a second image taken by the second camera, wherein a first angle of a first virtual line connecting the optical center of the first camera and a predetermined position on the object side photographed by the first camera with respect to a horizontal plane is determined by the optical angle of the second camera. The first camera and the second camera are arranged so that their angle with respect to the horizontal plane is different from a second angle of a second virtual line connecting the academic center and the specified position, the image processing device generates a first search area image from the first captured image and generates a second search area image from the second captured image, the first search area image has a larger area in the vertical direction than the second search area image, and the controller controls the height change mechanism to change the height positions of the first camera and the second camera when the image processing device detects the boarding and alighting area from the first search area image but does not detect the boarding and alighting area from the second search area image.

According to the present disclosure, the position of an aircraft's boarding and disembarking area can be accurately detected in accordance with various gate configurations using images captured by a camera installed at the tip of a passenger boarding bridge.

FIG. 1 is a schematic plan view showing an example of a passenger boarding bridge to which a detection system according to this embodiment is applied. FIG. 2 is a diagram showing the positional relationship between the first camera and the second camera in this embodiment. FIG. 3 is a block diagram showing a schematic configuration of a detection system applied to the passenger boarding bridge shown in FIG. FIG. 4 is a flowchart showing the flow of the search process in this embodiment. FIG. 5 is a flowchart showing the flow of the search process in this embodiment. FIG. 6 is a diagram showing an example of a captured image in this embodiment. FIG. 7 is a conceptual diagram for setting the first and second regions in the photographed image shown in FIG . FIG. 8 is a diagram showing an image after projective transformation of the first region in the first captured image in this embodiment, compared with an image before projective transformation. FIG. 9 is a diagram showing an image after projective transformation of the second region in the second captured image in this embodiment, compared with an image before projective transformation. FIG. 10 is a diagram illustrating an example of how the shooting areas in the first search area image and the second search area image overlap in this embodiment. FIG. 11 is a diagram showing the positional relationship between the first camera and the second camera in the first modification. FIG. 12 is a diagram showing the positional relationship between the first camera and the second camera in the second modification.

Preferred embodiments of the present disclosure will be described below with reference to the drawings. Note that, hereinafter, identical or corresponding elements will be designated by the same reference numerals throughout the drawings, and redundant explanations will be omitted. Furthermore, the drawings show each component diagrammatically to facilitate understanding, and shapes, dimensional ratios, etc. may not be accurately depicted. Furthermore, the present disclosure is not limited to the following embodiments.

Figure 1 is a schematic plan view showing an example of a passenger boarding bridge to which the detection system of this embodiment is applied.

The passenger boarding bridge 11 comprises a rotunda (base circular chamber) 4 connected to the boarding and disembarking entrance of the airport terminal building 2, a tunnel section 5 whose base end is connected to the rotunda 4 and is configured to be freely elevated and extendable in the longitudinal direction, and a cab (tip circular chamber) 6 connected to the tip of the tunnel section 5. The rotunda 4 is configured to be freely rotatable horizontally around a first rotation axis R4 extending vertically. The cab 6 is configured to be freely rotatable horizontally around a second rotation axis R6 extending perpendicular to the bottom surface of the cab 6.

The tunnel section 5 forms a walkway for passengers, and is configured with multiple cylindrical tunnels 5a, 5b nested within each other, allowing it to freely expand and contract in the longitudinal direction. While the tunnel section 5 is illustrated here as being composed of two tunnels 5a, 5b, it is acceptable for the tunnel section 5 to be composed of two or more tunnels. Furthermore, the base end of the tunnel section 5 is connected to the rotunda 4 so that it can swing freely (up and down) around a horizontal rotation axis, allowing it to be connected to the rotunda 4 so that it can be freely raised and lowered.

A drive column 7 is provided as a support leg at the tip of the tunnel section 5 (tunnel 5a at the tip side). The drive column 7 may also be attached to the cab 6. The drive column 7 is provided with a lifting device 8 that raises and lowers the cab 6 and tunnel section 5. This allows the cab 6 and tunnel section 5 to swing up and down with the rotunda 4 as the base point. As a result, the orientation of the second rotation axis R6, which is the rotation axis of the cab 6, changes depending on the inclination of the bottom surface of the cab 6 with respect to the horizontal plane. In other words, the second rotation axis R6 extends in a direction intersecting the horizontal plane, and its direction changes depending on the up and down swing of the cab 6 and tunnel section 5.

The drive column 7 also includes a travel device 9 for rotating the tunnel section 5 around the rotunda 4 (around the first rotation axis R4) and extending or retracting the tunnel section 5. The travel device 9 is provided below the lifting device 8. The travel device 9 has, for example, two travel wheels that can be driven independently in forward and reverse directions. Forward or reverse travel is possible by rotating the two travel wheels in the same direction (forward or reverse), and the travel direction (direction of the travel wheels) can be changed on the spot by rotating the two travel wheels in opposite directions. Note that this embodiment illustrates a configuration in which the tunnel section 5 rotates around the first rotation axis R4 as the rotunda 4 itself rotates around the first rotation axis R4. However, instead, the rotunda 4 may be fixed to the terminal building 2, and the tunnel section 5 may be connected to the rotunda 4 so that it can rotate freely around the first rotation axis R4.

The cab 6 has a connection part 6a at its tip that connects to the boarding and disembarking areas D1, D2 of the aircraft 3. The connection part 6a is equipped with a closure, a bumper, a distance sensor, etc. The position of the connection part 6a (the direction of connection to the boarding and disembarking areas D1, D2) can be changed by rotating the cab 6 about the second rotation axis R6. Note that instead of rotating the cab 6, the connection part 6a may rotate about the second rotation axis R6 relative to the cab 6.

Cabs 6 are also equipped with cameras 21a and 21b for photographing the sides of the aircraft 3. Cameras 21a and 21b are provided at the front end of the cab 6. In the example of Figure 1, the first camera 21a is installed at the top inside of the connection 6a in the cab 6. The second camera 21b is installed at the bottom inside of the connection 6a in the cab 6. In other words, the first camera 21a is positioned higher than the second camera 21b. The first camera 21a is also positioned such that its longitudinal position in the cab 6 (first distance K1 from the connection 6a shown in Figure 2) is the same as the longitudinal position of the second camera 21b in the cab 6 (second distance K2 from the connection 6a shown in Figure 2).

The positions of the first camera 21a and the second camera 21b may be other than inside the cab 6, as long as they are at a position at the tip of the cab 6 where the side of the aircraft 3 can be photographed. For example, the first camera 21a and the second camera 21b may be installed on the upper and lower exterior parts of the cab 6.

In this embodiment, the positional relationship between the first camera 21a and the second camera 21b is determined as follows. Figure 2 is a diagram showing the positional relationship between the first camera 21a and the second camera 21b in this embodiment. As shown in Figure 2, the first camera 21a and the second camera 21b are disposed so that a first angle δ1, relative to the horizontal plane H, of a first virtual line V1 connecting the optical center of the first camera 21a and a predetermined position VO on the object side to be photographed by the first camera 21a is different from a second angle δ2, relative to the horizontal plane H, of a second virtual line V2 connecting the optical center of the second camera 21b and the predetermined position VO.

The predetermined position VO on the object side may be set, for example, to a predetermined three-dimensional position in a three-dimensional coordinate system with the first camera 21a as the origin. Alternatively, the predetermined position VO on the object side may be set, for example, to a predetermined three-dimensional position in a three-dimensional coordinate system with a predetermined position on the apron EP as the origin. The three-dimensional position of the predetermined position VO may be set based on the detection position (the reference point of the door described below) of the boarding and alighting section D1 of the aircraft 3 when the boarding and alighting section D1 is located directly in front of the connecting section 6a. In this embodiment, the predetermined position VO is set at a position below the cab 6, so that the first angle δ1 is greater than the second angle δ2.

In this embodiment, the boarding and disembarking area D1 of the aircraft 3 to which the passenger boarding bridge 11 should connect is searched for based on the first captured image G1 taken by the first camera 21a installed at the tip of the passenger boarding bridge 11 and the second captured image G2 taken by the second camera 21b.

Figure 3 is a block diagram showing the schematic configuration of a detection system applied to the passenger boarding bridge shown in Figure 1. As shown in Figure 3, the detection system 20 in this embodiment comprises the above-mentioned cameras 21a and 21b, and an image processing device 23 that detects the boarding and disembarking section D1 of the aircraft 3 from the images G1 and G2 captured by the cameras 21a and 21b. The detection system 20 further comprises a memory 24 that stores data such as the captured images and image processing programs, and an output device 25 that outputs detection results, etc. These components 21a, 21b, 23, 24, and 25 can exchange data with each other via a bus 26.

The bus 26 is also connected to a controller 30 that controls the passenger boarding bridge 11. The controller 30 is provided, for example, in the cab 6 or the most distal tunnel 5a. The controller 30 controls the rotation of the cab 6 of the passenger boarding bridge 11, and the elevation and travel of the drive column 7. Furthermore, the controller 30 controls the rotation of the height change mechanism 36 and the shooting direction change mechanism 34 of the cameras 21a and 21b.

The height positions of the cameras 21a and 21b can be changed. To this end, the detection system 20 is equipped with a height change mechanism 36 that can change the height positions of the cameras 21a and 21b. In this embodiment, the height change mechanism 36 is the lifting device 8 of the drive column 7 that raises and lowers the cab 6. In other words, the height of the cameras 21a and 21b is changed together with the cab 6 to which they are attached. Alternatively, the height change mechanism 36 may be configured to change the height of the cameras 21a and 21b relative to the cab 6. For example, the cameras 21a and 21b may be attached to the outside of the cab 6 via height adjustment rails extending in the height direction, and the height change mechanism 36 may be able to adjust the positions of the cameras 21a and 21b on the height adjustment rails.

The cameras 21a and 21b are capable of changing the shooting direction (the direction of the shooting center axis L1) within the horizontal plane. To this end, the detection system 20 is equipped with a shooting direction change mechanism 34 that changes the shooting direction of the cameras 21a and 21b by rotating around a predetermined rotation axis that extends in a direction intersecting the horizontal plane.

The shooting direction change mechanism 34 may be configured as a camera rotation mechanism that rotates the cameras 21a and 21b relative to the cab 6. In this case, the shooting direction change mechanism 34 can change the shooting directions of the first camera 21a and the second camera 21b independently of each other. Alternatively, the shooting direction change mechanism 34 may be the cab 6. For example, the cameras 21a and 21b may be fixed to the cab 6, and the shooting direction (the orientation of the shooting center axis L1) of the cameras 21a and 21b may be changed by rotating the cab 6 around a second rotation axis R6 that extends in a direction intersecting the horizontal plane.

The controller 30 and image processing device 23 are configured by a computer that performs various calculations and processing based on the data stored in the memory device 24. For example, the controller 30 and image processing device 23 are equipped with a CPU, main memory (RAM), a communication interface, etc. The controller 30 and image processing device 23 may be configured by the same computer, or may be configured by different computers.

The output device 25 outputs the results of calculations or processing performed by the controller 30 and the image processing device 23. The output device 25 may be configured, for example, as a monitor that displays the calculation results or a communicator that transmits data to a server or a communication terminal via a communication network. The image processing device 23 and memory device 24 of the detection system 20 may be configured as a server connected to the cameras 21a, 21b via a communication network. That is, the server may acquire images captured by the cameras 21a, 21b, perform the search processing described below on the server, and display the results on a monitor or a display unit of a communication terminal installed in the control room of the passenger boarding bridge 11, for example.

The controller 30 and image processing device 23 grasp the position coordinates of each part of the passenger boarding bridge 11 in real time using a predetermined three-dimensional coordinate system, such as an XYZ Cartesian coordinate system. For example, the position coordinates of each part of the passenger boarding bridge 11 are expressed as absolute coordinates in a three-dimensional coordinate system with the intersection of the first rotation axis R4 of the rotunda 4 and the plane of the apron EP as the origin (0,0,0) and the mutually orthogonal X, Y, and Z axes. The X, Y, and Z coordinate values of these position coordinates each indicate the distance (e.g., in mm) from the origin (0,0,0), which is the position of the first rotation axis R4 of the rotunda 4. The controller 30 and image processing device 23 express the positions of each part of the aircraft 3 and the passenger boarding bridge 11 as position coordinates using such a three-dimensional Cartesian coordinate system.

The image processing device 23 has a search area image generation unit 31, a search execution unit 32, and a search range determination unit 33 as functional blocks to perform the search processing described below. These functional blocks partially or entirely include circuits, including integrated circuits. Therefore, these configurations 31, 32, and 33 can be considered circuits. These functional blocks are hardware that performs the enumerated functions, or hardware programmed to perform the enumerated functions. The hardware may be hardware disclosed in this specification, or other known hardware that is programmed or configured to perform the enumerated functions. If the hardware is a processor, which is considered a type of circuit, these functional blocks are a combination of hardware and software, and the software is used to configure the hardware or processor. The image processing device 23 reads a program for performing the search processing stored in the memory device 24 and performs the search processing described below.

The search process for the boarding and disembarking section D1 in this embodiment is described below. Figures 4 and 5 are flowcharts showing the flow of the search process in this embodiment. The search process is performed when the aircraft 3 arrives at a predetermined arrival position. For this reason, the passenger boarding bridge 11 is located in an initial position (standby position) that avoids the aircraft 3, as shown in Figure 1.

The memory 24 stores data on the search ranges that are preset for the first camera 21a and the second camera 21b. The search ranges are set as areas along the aircraft axis AL direction. The memory 24 also stores the initial angles (photography direction data) of the cameras 21a and 21b. The initial angles of the cameras 21a and 21b are set to an angle at the initial position of the passenger boarding bridge 11 such that at least one end of the predetermined search range in the aircraft axis AL direction is included within the angle of view of the cameras 21a and 21b.

Here, the search process in this embodiment is performed by sequentially moving from the front to the rear in the direction of the aircraft axis AL of the aircraft 3. Therefore, the initial angles of the cameras 21a and 21b are set to an angle at the initial position of the passenger boarding bridge 11 such that the front end of the search range set for the cameras 21a and 21b in the direction of the aircraft axis AL is included within the angle of view of the cameras 21a and 21b.

First, in the search process, the controller 30 controls the shooting direction change mechanism 34 so that the angles of the cameras 21a and 21b are set to the initial angle (step S1). Furthermore, the controller 30 controls the height change mechanism 36 so that the heights of the cameras 21a and 21b are set to the initial height. The cameras 21a and 21b each capture images at their initial angles (step S2). The image processing device 23 acquires a first captured image G1 captured by the first camera 21a at the initial angle and a second captured image G2 captured by the second camera 21b at the initial angle. The acquired captured images G1 and G2 are stored in the memory 24. Note that the image processing device 23 may perform predetermined image processing on the captured images G1 and G2, such as correcting distortion or brightness due to the camera lens. Positions in the captured images G1 and G2 are expressed as position coordinates in a two-dimensional coordinate system with one vertex of the captured images G1 and G2 (e.g., the upper left vertex) as the origin and perpendicular U- and V-axes.

The search area image generation unit 31 divides a first area Eai (i = 1, 2, 3, ...) that includes a portion of the first captured image G1, and generates a first search area image Cai (see Figure 8, described later) that includes a portion of the first captured image G1 that is partitioned based on the first area Eai. To achieve this, the search area image generation unit 31 first sets a first area Eai in the first captured image G1 that corresponds to a virtual rectangular frame Bi in three-dimensional space, which is real space (step S3). The initial first area Ea1 in the search process is set to an area that includes the front end of the search range in the direction of the axis AL. Similarly, the search area image generation unit 31 divides a second area Ebi (i = 1, 2, 3, ...) that includes a portion of the second captured image G2, and generates a second search area image Cbi (see Figure 9, described later) that includes a portion of the second captured image G2 that is partitioned based on the second area Ebi.

Figure 6 is a diagram showing an example of a captured image in this embodiment. Figure 7 is a conceptual diagram for setting the first and second regions in the captured image shown in Figure 6. While Figures 6, 7, and the following description primarily describe the manner in which the first search area image Cai is generated from the first captured image G1, the manner in which the second search area image Cbi is generated from the second captured image G2 is similar. That is, the manner in which the second search area image Cbi is generated from the second captured image G2 is shown by replacing the first camera 21a, first captured image G1, first region Eai, and first search area image Cai with the second camera 21b, second captured image G2, second region Ebi, and second search area image Cbi, respectively, in Figures 6, 7, and the following description. The first region Eai and the second region Ebi are set in the same region in real space (as regions corresponding to the same virtual rectangular frame Bi).

In this embodiment, the search area image generation unit 31 sets a virtual plane VP in the first captured image G1 that is perpendicular to the horizontal plane (apron EP) and parallel to the aircraft axis AL. Here, the aircraft axis AL is set in advance at a position a predetermined distance above the marshal line 13, assuming that the aircraft 3 will be parked on the marshal line 13 indicated on the apron EP. Note that, in this specification and claims, the forward and aft sides of the aircraft axis AL refer to the forward and aft sides of the aircraft 3 parked on the marshal line 13.

The virtual plane VP is set at a position offset by an offset amount W from the aircraft axis AL to one side in the aircraft width direction (the side closer to the passenger boarding bridge 11). In other words, the virtual plane VP is defined as a plane that includes a reference axis VL offset by an offset amount W from the aircraft axis AL to one side in the aircraft width direction and is perpendicular to the horizontal plane. The offset amount W is set in advance, taking into account the aircraft width of the aircraft 3, so that the distance between the virtual plane VP and the outer surface of the boarding and disembarking area D1 is close (less than a predetermined distance).

The search area image generation unit 31 sets a virtual rectangular frame Bi in three-dimensional space on the virtual plane VP. For example, the virtual rectangular frame Bi may be configured as a square frame with sides measuring 4 m on the virtual plane VP, centered at a reference point Pi on the reference axis VL. Because the shooting direction (shooting center axis L1) of the first camera 21a is often not perpendicular to the virtual plane VP, the virtual rectangular frame Bi often has a distorted quadrilateral shape on the first captured image G1, as shown by the first area Eai in Figure 6. There is a one-to-one correspondence between the three-dimensional coordinates Bi (xi, yi, zi) in real space and the two-dimensional coordinates Eai (ui, vi) on the first captured image G1 captured by the first camera 21a. The transformation between the three-dimensional coordinate system in real space and the two-dimensional coordinate system on the first captured image G1 is performed using perspective projection transformation.

It is desirable that the rectangular shape (first area) Eai on the first captured image G1 corresponding to the virtual rectangular frame Bi is entirely contained within the captured image G1, but it is sufficient that at least a portion of the rectangular shape is captured in the first captured image G1.

The search area image generation unit 31 performs a projective transformation on the virtual plane VP in the first area Eai (an area including a part of the first captured image G1) set in this manner so that the image becomes what it would appear when viewed from a direction perpendicular to the virtual plane VP (step S4).

Here, projective transformation will be explained in more detail. Figure 8 is a diagram showing an image after projective transformation of a first region in a first captured image in this embodiment, compared with the image before projective transformation. Also, Figure 9 is a diagram showing an image after projective transformation of a second region in a second captured image in this embodiment, compared with the image before projective transformation. Figures 8 and 9 illustrate a case where a virtual rectangular frame Bi is set near the boarding and disembarking area D1. Also, in Figures 8 and 9, a portion of the outer surface of the aircraft 3 is indicated by a two-dot chain line.

As described above, the first area Eai, which includes a portion of the first captured image G1, often has a distorted rectangular shape. In the first search area image Cai, which is the image after projective transformation, the virtual rectangular frame Bi has the same rectangular shape as the shape set in three-dimensional space. For example, if the virtual rectangular frame Bi is square, the first area Eapi corresponding to the virtual rectangular frame Bi after projective transformation will also be square. Accordingly, in the image after projective transformation, a portion of the aircraft 3 close to the virtual plane VP is corrected to approach the image viewed from a direction perpendicular to the aircraft axis AL of the aircraft 3. Note that since only objects on the virtual plane VP are strictly corrected, the further away from the virtual plane VP an object is, the greater the correction error will be.

In Figure 8, the shooting direction (center axis L1) of the first camera 21a is tilted with respect to the vehicle axis AL. Therefore, the door, which is the access point D1 included in the first area Eai, appears with its lower and upper ends tilted with respect to the horizon in the first captured image G1. As will be described later, when searching for the access point D1 (using the aforementioned AI image recognition, etc.), the reference point for the door is set, for example, on the door sill or a reinforcing plate provided on the door sill. In this case, in the image used to search for the access point D1, it is preferable that the lower end (door sill or reinforcing plate) of the access point D1 (door) extend horizontally.

In this embodiment, as shown in FIG. 8, a first region Eai including a portion of the first captured image G1 is projectively transformed so that a virtual rectangular frame Bi set on a virtual plane VP parallel to the vehicle axis AL and perpendicular to the horizontal plane (apron EP) is displayed as a rectangle. Furthermore, projective transformation suppresses changes in the size of the boarding and alighting area D1 due to the distance between the first camera 21a and the boarding and alighting area D1. In other words, the size of the boarding and alighting area D1 after projective transformation can be made approximately the same regardless of the distance between the first camera 21a and the boarding and alighting area D1. Therefore, the boarding and alighting area D1 can be detected accurately in the image after projective transformation.

As shown in Figure 9, for the second captured image G2 captured by the second camera 21b, a second region Ebi including a portion of the second captured image G2 is similarly projectively transformed so that a virtual rectangular frame Bi set on a virtual plane VP parallel to the camera axis AL and perpendicular to the horizontal plane (apron EP) is displayed as a rectangle.

The search area image generation unit 31 adjusts the search area images Cai and Cbi by cropping the image after the projective transformation (step S5). The search area image generation unit 31 crops the image after the projective transformation so that the first search area image Cai, which is a partial area of the first captured image G1, has a larger area in the vertical direction than the second search area image Cbi, which is a partial area of the second captured image G2.

In this embodiment, as shown in Figure 8, the first search area image Cai includes the entire first area Eapi after projective transformation, and is set to an area where the length Wah between a pair of horizontal boundary lines Bah that define the vertical direction is longer than the length Wav between a pair of vertical boundary lines Bav that define the left-right direction.On the other hand, as shown in Figure 9, the second search area image Cbi includes the entire second area Ebpi after projective transformation, and is set to an area where the length Wbh between a pair of horizontal boundary lines Bbh that define the vertical direction is shorter than the length Wbv between a pair of vertical boundary lines Bbv that define the left-right direction.

Here, the vertical length Wah of the first search area image Cai is longer than the vertical length Wbh of the second search area image Cbi. Furthermore, the horizontal length Wbv of the second search area image Cbi is longer than the horizontal length Wav of the first search area image Cai. By trimming in this manner, even when using images G1 and G2 captured by the first camera 21a and the second camera 21b, which have the same angle of view, the first search area image Cai can be made to have a vertically larger area than the second search area image Cbi, making it easier to determine whether the height position is appropriate.

The search area image generation unit 31 then performs a predetermined extrapolation process on the adjusted image. For example, the extrapolation process may include filling pixels outside the range of the captured images G1 and G2 with a single color (e.g., black), copying pixels at the edges, etc. The generated search area images Cai and Cbi are also enlarged so that the entire virtual rectangular frame Bi appears as large as possible in the search area images Cai and Cbi.

The search execution unit 32 searches for each of the search area images Cai and Cbi generated as described above to determine whether there is a door within the search area image Cai or Cbi that is a candidate for the boarding/alighting area D1 (step S6). The door search method is not particularly limited as long as it is an image recognition process that can detect doors, but for example, AI image recognition using a trained model generated by deep learning may be used.

In the door image recognition process, the door and its reference point are detected based on the painted outline of the door and the shape of the reinforcing plate attached to the door sill. The door reference point is set, for example, at the center of the door sill or the center of the reinforcing plate.

If the search execution unit 32 detects a door in the first search area image Cai and a door in the second search area image Cbi (Yes in both steps S7 and S8), it calculates the three-dimensional coordinates of the door's reference point (step S9). The three-dimensional coordinates of the door can be obtained, for example, by converting the two-dimensional coordinates in each search area image Cai and Cbi into two-dimensional coordinates in the original captured images G1 and G2, respectively, and then converting the two-dimensional coordinates in the first captured image G1 and the two-dimensional coordinates in the second captured image G2 into three-dimensional coordinates in real space. Note that the three-dimensional coordinates in real space may be calculated from the two-dimensional coordinates of only one of the search area images Cai and Cbi. In this case, for example, the calculation may be performed by approximating the door's reference point to be on the virtual plane VP.

The search execution unit 32 determines whether the detected door is the boarding/alighting section D1 to which the passenger boarding bridge 11 should be attached based on the three-dimensional coordinates of the detected door (step S10). A search range is stored in advance in the memory 24 to include all of the doors of various aircraft to which the passenger boarding bridge 11 should be attached. If the three-dimensional coordinates of the detected door are within the search range stored in the memory 24, the search execution unit 32 determines that the boarding/alighting section D1 has been detected. If it is determined that the boarding/alighting section D1 has been detected (Yes in step S10), the search execution unit 32 outputs a result of successful search (step S11).

If a door is detected in the first search area image Cai but not in the second search area image Cbi (Yes in step S7, No in step S8), the controller 30 controls the height change mechanism 36 to change the height positions of the first camera 21a and the second camera 21b.

In this embodiment, as described above, the first camera 21a is installed at the upper inside of the connection portion 6a of the cab 6. The second camera 21b is installed at the lower inside of the connection portion 6a of the cab 6. The door reference point for searching the boarding/alighting area D1 is set at the door sill located at the lower end of the door or a reinforcing plate provided on the door sill. Therefore, if the height position of the second camera 21b, which is located at the lower inside of the connection portion 6a of the cab 6, is located higher than the height position of the door reference point at the boarding/alighting area D1 of the aircraft 3, as shown in FIG. 9, the lower part of the photographing range of the second camera 21b (second photographed image G2) may be blocked by the floor surface X of the cab 6, and the door reference point may not be captured.

Therefore, if a door is detected in the first search area image Cai generated based on the first captured image G1 taken by the first camera 21a, but not in the second search area image Cbi generated based on the second captured image G2 taken by the second camera 21b, it is considered that the height position of the second camera 21b is located above the height position of the reference point of the door at the boarding and disembarking area D1 of the aircraft 3.

The controller 30 therefore controls the height change mechanism 36 to change the height positions of the first camera 21a and the second camera 21b to lower positions. In this embodiment, the controller 30 lowers the cab 6 a predetermined distance using the lifting device 8 of the drive column 7. The amount of height change is set so that the images captured before and after the height change partially overlap. This eliminates the need to provide a separate mechanism for changing the height positions of the first camera 21a and the second camera 21b, and also reduces the amount of adjustment required to align the height position of the passenger boarding bridge 11 with the boarding and disembarking section D1 when connecting the connection section 6a on the passenger boarding bridge 11 side to the boarding and disembarking section D1 of the aircraft 3.

Here, since the first search area image Cai has a larger area in the vertical direction than the second search area image Cbi, the first search area image Cai can be an image in which, if a door is included within the horizontal range of the first search area image Cai, the reference point of the door is located within the first search area image Cai.

After the height change, cameras 21a and 21b capture images again (step S2). Subsequently, the above-described search process is similarly performed on the captured images G1 and G2 after the height change (steps S3 to S11).

If a door is not detected in the first search area image Cai (No in step S7) or if it is determined that the detected door is not the boarding/disembarking area D1 (No in step S10), the search area image generation unit 31 moves the first area Eai from one side to the other in the direction of the aircraft's axis AL to sequentially generate first search area images Cai, and moves the second area Ebi from one side to the other in the direction of the aircraft's axis AL to sequentially generate second search area images Cbi. In this embodiment, the search area image generation unit 31 moves the first area Eai and the second area Ebi from the front side to the rear side in the direction of the aircraft's axis AL.

To achieve this, the search area image generation unit 31 first shifts (moves) the virtual rectangular frame Bi a predetermined distance from one side to the other in the direction of the aircraft 3's axis AL in three-dimensional space. The search area image generation unit 31 calculates a first area Eai in the first photographed image G1 and a second area Ebi in the second photographed image G2 that correspond to the shifted virtual rectangular frame Bi. In this way, the search area image generation unit 31 shifts (moves) the first area Eai in the first photographed image G1 from one side to the other in the direction of the aircraft 3's axis AL, and shifts (moves) the second area Ebi in the second photographed image G2 from one side to the other in the direction of the aircraft 3's axis AL (step S13).

For example, in three-dimensional space, a reference point Pi+1 is set at a position on the reference axis VL a predetermined distance rearward from the position of the reference point Pi in the direction of the axis AL, and a virtual rectangular frame Bi+1 is set with reference point Pi+1 as its base. The distance between reference points Pi and Pi+1 is preferably equal to or less than the length of one horizontal side of the virtual rectangular frame Bi, and more preferably between 1/4 and 1/2 of the length of that side. This allows for multiple opportunities to detect the same door (two or three times or so) during door search.

The search range determination unit 33 determines whether the shifted first region Eai and second region Ebi are within a predetermined search range, i.e., whether the entire search range has been searched (step S14). When determining whether the shifted first region Eai and second region Ebi are within the search range, the search range determination unit 33 may also determine whether the corresponding virtual rectangular frame Bi in three-dimensional space is within the search range. If the shifted first region Eai and second region Ebi are within the search range (Yes in step S14), the search range determination unit 33 further determines whether the shifted first region Eai is within the range of the first captured image G1 used in the previous search, and whether the shifted second region Ebi is within the range of the second captured image G2 used in the previous search (step S15).

If the shifted first area Eai is within the range of the first captured image G1 and the shifted second area Ebi is within the range of the second captured image G2 (Yes in step S15), the search area image generation unit 31 sets (updates) the shifted first area Eai as the first area Eai for generating the first search area image Cai, and sets (updates) the shifted second area Ebi as the second area Ebi for generating the second search area image Cbi (step S3).

The search area image generation unit 31 generates (updates) the first search area image Cai and the second search area image Cbi in the same manner as described above based on the updated first area Eai and second area Ebi. The search execution unit 32 searches for the boarding and disembarking area D1 in the updated first search area image Cai and second search area image Cbi. In this manner, the search execution unit 32 repeatedly searches for the boarding and disembarking area D1 using the multiple first search area images Cai generated sequentially while changing the first search area image Cai by shifting the first area Eai from the front to the rear along the vehicle axis AL. Similarly, the search execution unit 32 repeatedly searches for the boarding and disembarking area D1 using the multiple second search area images Cbi generated sequentially while changing the second search area image Cbi by shifting the second area Ebi from the front to the rear along the vehicle axis AL.

If the shifted first area Eai is within the search range but outside the range of the first captured image G1, or if the shifted second area Ebi is within the search range but outside the range of the second captured image G2 (No in step S15), the search range determination unit 33 sends an instruction signal to the controller 30 to change the shooting direction of the cameras 21a and 21b. The controller 30 controls the corresponding shooting direction change mechanism 34 to change the shooting direction of the cameras 21a and 21b based on the instruction signal (step S16). The amount of change in the shooting direction is set so that the captured images before and after the shooting direction change partially overlap each other.

In the example of Figure 7, by rotating the cab 6 by a predetermined angle θ around the second rotation axis R6, the imaging center axis L1 of the cameras 21a and 21b is changed to an imaging center axis L1a that intersects with the reference axis VL of the virtual plane VP rearward in the direction of the vehicle axis AL compared to the imaging center axis L1. Note that, as mentioned above, the method of changing the imaging direction of the cameras 21a and 21b is not limited to this; for example, the first camera 21a and the second camera 21b themselves may be rotated around a predetermined rotation axis that extends in a direction intersecting the horizontal plane relative to the cab 6.

The first camera 21a and the second camera 21b take images again after changing the shooting direction (step S2). Thereafter, a search process is similarly performed on the updated captured images (steps S3 to S11).

If the first area Eai or the second area Ebi after the shift falls outside the search range (No in step S14), or if the number of retries is less than a preset reference value (No in step S17), the search range determination unit 33 increments the number of retries by one and restarts the search process from scratch. That is, the cameras 21a and 21b are returned to their initial angles to take photographs, and a search is performed based on the captured images G1 and G2. If the number of retries is equal to or greater than the reference value (Yes in step S17), the search execution unit 32 determines that the boarding and disembarking area D1 could not be detected and outputs a search failure result (step S18).

As described above, according to this embodiment, a first search area image Cai is generated from the image captured by the first camera 21a to detect and determine the boarding and disembarking area D1 of the aircraft 3, and a second search area image Cbi is generated from the image captured by the second camera 21b to detect and determine the boarding and disembarking area D1 of the aircraft 3.

If the boarding and alighting area D1 is detected in the first search area image Cai but not in the second search area image Cbi, the height positions of the first camera 21a and the second camera 21b are changed. Since the first search area image Cai has a larger area in the vertical direction than the second search area image Cbi, if the boarding and alighting area D1 can be detected in the first search area image Cai but not in the second search area image Cbi, the heights of the first camera 21a and the second camera 21b are not appropriate.

With the above configuration, even if the height positions of the first camera 21a and the second camera 21b are not appropriate for detecting the boarding and disembarking area D1 of the aircraft 3 due to the relative positions of the aircraft 3 and the passenger boarding bridge 11, the height positions of the first camera 21a and the second camera 21b are automatically changed. This allows the height position of the search area for the boarding and disembarking area D1 to be automatically changed to match the position of the boarding and disembarking area D1 of the aircraft 3. Therefore, the position of the boarding and disembarking area D1 of the aircraft 3 can be accurately detected in accordance with various gate arrangements.

Furthermore, if the boarding and disembarking area D1 is not detected from the first search area image Cai, it is determined that the boarding and disembarking area D1 does not exist in the current search area (position of the virtual rectangular frame Bi) in the direction of the aircraft axis AL of the aircraft 3, and the areas Eai and Ebi are shifted.

Figure 10 is a diagram illustrating the overlap of the captured areas in the first search area image and the second search area image in this embodiment. In the example shown in Figure 10, the judgment area including the first search area image Cai and the second search area image Cbi includes the first judgment area Q1, the second judgment area Q2, and the third judgment area Q3.

The first judgment area Q1 is the area where the upper area of the first search area image Cai and the left-right central area of the second search area image Cbi overlap. If a boarding/alighting area D1 is detected in the first judgment area Q1, it is determined that the boarding/alighting area D1 to be searched for has been detected. The second judgment area Q2 is the lower area of the first search area image Cai and is an area that does not overlap with the second search area image Cbi. If a boarding/alighting area D1 is detected in the second judgment area Q2, it is determined that a height adjustment of the cameras 21a, 21b is necessary. The third judgment area Q3 is an area on both left-right ends of the second search area image Cbi and is an area that does not overlap with the first search area image Cai. If a boarding/alighting area D1 is detected in the third judgment area Q3 or if a boarding/alighting area D1 is not detected in any of the judgment areas Q1, Q2, or Q3, it is determined that a shift of the search area in the direction of the axis AL is necessary.

In this way, by branching the flow of the search process based on whether the boarding and disembarking area D1 is detected in any of the judgment areas Q1, Q2, Q3 contained in the first search area image Cai and the second search area image Cbi, or whether it is not detected in any of the judgment areas Q1, Q2, Q3, it is possible to automate the decision on how to change the search area.

Furthermore, from each captured image G1, G2 taken by cameras 21a, 21b installed at the tip of the passenger boarding bridge 11, multiple search area images Cai, Cbi are generated, each divided into areas Eai, Ebi that include a portion of the captured image G1, G2. The multiple search area images Cai, Cbi are generated by moving the areas Eai, Ebi from one side to the other in the direction of the aircraft's axis AL of the aircraft 3. Therefore, by repeatedly searching for the boarding and disembarking area D1 while changing the search area images Cai, Cbi using the multiple search area images Cai, Cbi generated sequentially, it is possible to search for the boarding and disembarking area D1 in the entire captured images G1, G2 while moving the search area in the direction along the aircraft's axis AL of the aircraft 3.

This allows a uniform search for the boarding and disembarking section D1 regardless of the type of aircraft 3 or the gate arrangement. This makes it possible to correctly detect the boarding and disembarking section D1 to be installed even in gate arrangements that could not be handled by conventional methods of detecting the position of the boarding and disembarking section D1 that rely solely on three-dimensional position. Furthermore, when searching for the boarding and disembarking section D1, it is possible to eliminate the need for cumbersome operations such as inputting information corresponding to the parked aircraft 3. In other words, the search process can be performed for various aircraft 3 simply by the operator operating a button to start the search process.

Furthermore, according to this embodiment, the search area images Cai and Cbi are generated as images parallel to the aircraft axis AL by projective transformation. Therefore, even if the captured images G1 and G2 are captured from an oblique direction of the aircraft 3, distortion of the shape of the boarding and disembarking area D1 to be searched for in the search area images Cai and Cbi can be suppressed. This improves the search accuracy of the boarding and disembarking area D1 in the search area images Cai and Cbi.

Furthermore, according to this embodiment, the search area image generation unit 31 performs projective transformation on the captured images G1, G2 in the areas Eai, Ebi each time the captured images G1, G2 are divided into areas Eai, Ebi that include a portion of the captured images G1, G2. In this way, the captured images G1, G2 are projectively transformed (and cropped) for each area Eai, Ebi, which can reduce the reduction in image resolution compared to projectively transforming the entire captured images G1, G2 and then dividing and enlarging the projectively transformed images into specific areas Eai, Ebi.

Furthermore, according to this embodiment, if it is determined that the shifted areas Eai and Ebi are within the search range but not within the captured images G1 and G2, the shooting direction change mechanism 34 changes the shooting direction of cameras 21a and 21b, and cameras 21a and 21b capture images again. In this way, by taking multiple images while changing the shooting direction of cameras 21a and 21b, it is possible to obtain multiple captured images G1 and G2 of the entire search range. Therefore, since the shooting range per captured image can be relatively narrow, the resolution of the search area images Cai and Cbi obtained from the captured images G1 and G2 can be increased. Alternatively, it is possible to set the search range wider than the field of view of cameras 21a and 21b.

[Modification]
Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments, and various improvements, changes, and modifications are possible within the scope of the spirit of the present disclosure.

For example, in the above embodiment, the first camera 21a and the second camera 21b have the same angle of view, and when generating the search area images Cai and Cbi from the captured images G1 and G2, the first search area image Cai is adjusted (by changing the crop shape) so that it has a larger area in the vertical direction than the second search area image Cbi. However, this is not limited to this. For example, the first camera 21a may have a wider angle of view than the second camera 21b. In this case, even if the first search area image Cai and the second search area image Cbi are generated from areas of the same area in the first captured image G1 and the second captured image G2, the area of real space included in the first search area image Cai will be larger than the area of real space included in the second search area image Cbi.

Furthermore, while the above embodiment illustrates an example in which the second search area image Cbi is adjusted to have a wider area in the left-right direction than the first search area image Cai, this is not limited to this. For example, the first search area image Cai and the second search area image Cbi may have the same width in the left-right direction. Furthermore, the aspect ratio (Wah:Wav) of the first search area image Cai is not limited to the above embodiment (Wah > Wav). Similarly, the aspect ratio (Wbh:Wbv) of the second search area image Cbi is not limited to the above embodiment (Wbv > Wbh).

In addition, in the above embodiment, when the boarding and alighting area D1 is not detected in the first search area image Cai, an example was given of the case where the search area images Cai and Cbi are shifted in the direction of the axis AL regardless of whether the boarding and alighting area D1 is detected in the second search area image Cbi (third judgment area Q3 in Figure 10).However, the subsequent processing may be different depending on whether the boarding and alighting area D1 is detected only in the second search area image Cbi or whether the boarding and alighting area D1 is not detected in either the first search area image Cai or the second search area image Cbi.

For example, the search area image generation unit 31 may shift the first area Eai and the second area Ebi in the direction of the axis AL differently when the boarding and alighting area D1 is detected only in the second search area image Cbi than when the boarding and alighting area D1 is not detected in either the first search area image Cai or the second search area image Cbi. In this case, for example, the shift amount of the first area Eai and the second area Ebi in the direction of the axis AL when the boarding and alighting area D1 is not detected in either the first search area image Cai or the second search area image Cbi may be larger than the shift amount when the boarding and alighting area D1 is detected only in the second search area image Cbi.

More specifically, if the boarding and alighting area D1 is detected only in the second search area image Cbi, i.e., if the boarding and alighting area D1 is detected in the third judgment area Q3, the search area image generation unit 31 may shift the first area Eai and the second area Ebi so that the boarding and alighting area D1 is located in the first judgment area Q1. Furthermore, if the boarding and alighting area D1 is not detected in either the first search area image Cai or the second search area image Cbi, the search area image generation unit 31 may shift the first area Eai and the second area Ebi so that the area outside the third judgment area Q3 in the direction of the vehicle axis AL is located in the first judgment area Q1.

In addition, the above embodiment illustrates an example in which the first camera 21a is located at the same fore-and-aft position in the cab 6 as the second camera 21b but at a higher position, but this is not limited to this. Figure 11 is a diagram showing the positional relationship between the first camera and the second camera in variant example 1. As shown in Figure 11, the first camera 21a and the second camera 21b may be positioned so that they are located at the same height but at different fore-and-aft positions in the cab 6. The fore-and-aft position of the first camera 21a in the cab 6 is located forward of the fore-and-aft position of the second camera 21b in the cab 6 (at a position closer to the connection portion 6a). In this case as well, the first angle δ1 of the first virtual line V1 connecting the optical center of the first camera 21a and the predetermined position VO on the object side photographed by the first camera 21a with respect to the horizontal plane H is different from the second angle δ2 of the second virtual line V2 connecting the optical center of the second camera 21b and the predetermined position VO with respect to the horizontal plane H (the first angle δ1 is larger than the second angle δ2). Furthermore, the first camera 21a and the second camera 21b may be disposed at different height positions and at different positions in the fore-and-aft direction in the cab 6.

In addition, in the above embodiment, the first camera 21a, which captures the first captured image G1 for generating the first search area image Cai, which has a wide area in the vertical direction, is positioned above (or in front of) the second camera 21b. However, as shown in FIG. 12, the first camera 21a may be positioned below (or behind) the second camera 21b. FIG. 12 is a diagram showing the positional relationship between the first camera and the second camera in Modification 2. The example in FIG. 12 differs from the above embodiment (example in FIG. 2) in that the positions of the first camera 21a and the second camera 21b are swapped. In the example in FIG. 12, the predetermined position VO is set above the cab 6. Therefore, in this example as well, the first angle δ1 is greater than the second angle δ2.

For example, if the door reference point is set at the upper end of the door, and the height position of the second camera 21b, which is located at the top inside of the connecting portion 6a of the cab 6, is lower than the height position of the door reference point at the boarding/disembarking area D1 of the aircraft 3, the upper part of the capturing range of the second camera 21b (second captured image G2) may be blocked by the ceiling of the cab 6, and the door reference point may not be captured. Therefore, if a door is detected in the first search area image Cai generated based on the first captured image G1 captured by the first camera 21a, but not in the second search area image Cbi generated based on the second captured image G2 captured by the second camera 21b, it is considered that the height position of the second camera 21b is lower than the height position of the door reference point at the boarding/disembarking area D1 of the aircraft 3. In this case, the controller 30 controls the height change mechanism 36 to change the height positions of the cameras 21a and 21b to higher positions.

In addition, while the above embodiment illustrates a mode in which a search is performed from the front to the rear in the direction of the aircraft axis AL of the aircraft 3, this is not limited to this, and a search may be performed from the rear to the front in the direction of the aircraft axis AL of the aircraft 3. For example, when connecting the connection section 6a of the passenger boarding bridge 11 to the boarding and disembarking section D2 located at the rear of the aircraft 3, it is preferable to perform a search from the rear to the front in the direction of the aircraft axis AL of the aircraft 3.

In addition, while the above embodiment illustrates a system including one passenger boarding bridge 11, the detection system 20 may include two or more passenger boarding bridges. For example, the detection system 20 disclosed herein can also be applied to a system including two passenger boarding bridges, a first boarding bridge and a second boarding bridge.

For example, the connection portion 6a of the cab 6 on the first boarding bridge is connected to the boarding/disembarking section D1 on the front side of the aircraft 3, and the connection portion 6a of the cab 6 on the second boarding bridge is connected to the boarding/disembarking section D2 on the rear side of the aircraft 3. In this case, in the search process for the boarding/disembarking section D1 connected to the first boarding bridge, the search is performed from the front to the rear in the direction of the aircraft axis AL of the predetermined search range for the first boarding bridge. That is, in the search process for the boarding/disembarking section D1 connected to the first boarding bridge, as described above, the areas Eai and Ebi are moved from the front to the rear in the direction of the aircraft axis AL on the captured images G1 and G2. Furthermore, if necessary, the shooting direction of the cameras 21a and 21b is changed from the front to the rear of the aircraft 3 and re-shooting is performed.

On the other hand, in the search process for boarding and disembarking section D2 connected to the second boarding bridge, a search is performed from the rear side to the front side in the direction of aircraft axis AL within a predetermined search range for the second boarding bridge. That is, in the search process for boarding and disembarking section D2 connected to the second boarding bridge, areas Eai and Ebi are moved from the rear side to the front side in the direction of aircraft axis AL on the captured images G1 and G2. Furthermore, if necessary, the shooting direction of cameras 21a and 21b is changed from the rear side to the front side of the aircraft 3 and re-shooting is performed.

This allows priority detection of the boarding and disembarking section D1 at the front of the aircraft 3 based on the images G1, G2 taken by the cameras 21a, 21b installed on the first boarding bridge, and priority detection of the boarding and disembarking section D2 at the rear of the aircraft 3 based on the images G1, G2 taken by the cameras 21a, 21b installed on the second boarding bridge. This reduces the possibility of detecting the same boarding and disembarking section for multiple boarding bridges. Therefore, it is possible to accurately and quickly search for the boarding and disembarking section D1 to which the first boarding bridge should connect and the boarding and disembarking section D2 to which the second boarding bridge should connect.

Furthermore, the detection system 20 of the present disclosure can also be applied to a system including, for example, three boarding bridges. In this case, it is possible that the aircraft 3 has boarding and disembarking areas on the first and second floors.

In this case, for example, to connect the first boarding bridge to the boarding section located forward in the AL direction on the first floor, a first search is performed from forward to rear in the AL direction within a first search range. Furthermore, to connect the second boarding bridge to the boarding section located rearward in the AL direction on the first floor, a second search is performed from rearward to forward in the AL direction within a second search range. Furthermore, to connect the third boarding bridge to the boarding section on the second floor, a third search is performed from forward to rear in the AL direction within a third search range set higher than the first search range in the first search.

In addition, the search for the boarding and disembarking section D2 to which the second boarding bridge should connect may also be performed from the front to the rear in the direction of the aircraft axis AL, similar to the search for the boarding and disembarking section D1 to which the first boarding bridge should connect.

In addition, in the above embodiment, an example was given in which one controller 30 controls the operation of all of the passenger boarding bridge 11, the shooting direction change mechanism 34, and the height change mechanism 36, but different controllers may be provided for some or all of these components 11, 34, and 36.

[Disclosure Summary]
Each of the following sections discloses a preferred embodiment.

[Item 1]
A detection system according to an aspect of the present disclosure is a detection system for detecting boarding and disembarking sections of an aircraft, the detection system comprising: a passenger boarding bridge connected to a terminal building; a first camera and a second camera provided on the passenger boarding bridge; a height changing mechanism capable of changing the height positions of the first camera and the second camera; a controller for controlling the operation of the height changing mechanism; and an image processing device for detecting the boarding and disembarking section of the aircraft from a first photographed image taken by the first camera and a second photographed image taken by the second camera, wherein a first angle of a first virtual line connecting the optical center of the first camera and a predetermined position on the object side photographed by the first camera with respect to a horizontal plane is The first camera and the second camera are arranged so that their angle with respect to the horizontal plane is different from a second angle of a second virtual line connecting the academic center and the specified position, the image processing device generates a first search area image from the first captured image and generates a second search area image from the second captured image, the first search area image has a larger area in the vertical direction than the second search area image, and the controller controls the height change mechanism to change the height positions of the first camera and the second camera when the image processing device detects the boarding and alighting area from the first search area image but does not detect the boarding and alighting area from the second search area image.

With the above configuration, a first search area image for detecting and determining whether an aircraft's boarding and disembarking areas is located is generated from an image captured by the first camera, and a second search area image for detecting and determining whether an aircraft's boarding and disembarking areas is located is generated from an image captured by the second camera. If a boarding and disembarking area is detected in the first search area image but not in the second search area image, the height positions of the first and second cameras are changed. Because the first search area image has a larger vertical area than the second search area image, if a boarding and disembarking area can be detected in the first search area image but not in the second search area image, the heights of the first and second cameras are inappropriate. With the above configuration, even if the height positions of the first and second cameras are inappropriate for detecting an aircraft's boarding and disembarking areas due to the relative positions of the aircraft and the passenger boarding bridge, the height positions of the first and second cameras are automatically changed. This allows the height position of the search area for boarding and disembarking areas to be automatically changed to match the position of the aircraft's boarding and disembarking areas. Therefore, the position of an aircraft's boarding and disembarking areas can be accurately detected in accordance with various gate arrangements.

[Item 2]
In the detection system of item 1, the image processing device divides a first area including a portion of the first captured image to generate a first search area image including a portion of the first captured image divided based on the first area, and divides a second area including a portion of the second captured image to generate a second search area image including a portion of the second captured image divided based on the second area, wherein the first area and the second area are set to the same area in real space, and the portion of the first captured image may have a larger area in the vertical direction than the portion of the second captured image.

With the above configuration, even if a first camera and a second camera having the same angle of view are used, it is possible to easily determine whether the height position is appropriate.

[Item 3]
In the detection system of item 2, if the image processing device does not detect the boarding and disembarking area from the first search area image, it may sequentially generate the first search area images by moving the first area on the first captured image from one side in the aircraft's axial direction to the other side, and sequentially generate the second search area images by moving the second area on the second captured image from one side in the aircraft's axial direction to the other side, and may repeatedly perform the search while changing the first search area images using the sequentially generated multiple first search area images, and may repeatedly search for the boarding and disembarking area while changing the second search area images using the sequentially generated multiple second search area images.

With the above configuration, a plurality of first search area images are generated from a first captured image taken by a first camera, each divided into first regions including a portion of the first captured image. Similarly, a plurality of second search area images are generated from a second captured image taken by a second camera, each divided into second regions including a portion of the first captured image. The plurality of first search area images and the plurality of second search area images are generated by moving the first region or the second region from one side of the aircraft's axis to the other. Therefore, by repeatedly searching for boarding and disembarking areas while changing the search area image using the plurality of sequentially generated search area images, it is possible to search for boarding and disembarking areas across the entire captured image while moving the search area in a direction along the aircraft's axis. This allows for a uniform search for boarding and disembarking areas regardless of the type of aircraft or gate arrangement. Therefore, the location of an aircraft's boarding and disembarking areas can be accurately detected in accordance with various gate arrangements.

[Item 4]
In the detection system of any one of items 1 to 3, the first camera may be disposed above the second camera.

[Item 5]
In the detection system of any one of items 1 to 4, the passenger boarding bridge may include a cab having a connection portion connected to the boarding and disembarking portion of the aircraft, and a lifting device that raises and lowers the cab, the first camera and the second camera may be provided in the cab, and the height change mechanism may be the lifting device.

The above configuration eliminates the need to provide a separate mechanism for changing the height positions of the first and second cameras, and reduces the amount of adjustment required to align the height position of the passenger boarding bridge with the boarding and disembarking section when connecting the connection section on the passenger boarding bridge to the boarding and disembarking section of the aircraft.

[Item 6]
The detection system of item 3 may include a shooting direction change mechanism that changes the shooting direction of the first camera and the second camera by rotating around a predetermined rotation axis that extends in a direction intersecting a horizontal plane, and the image processing device determines whether the first area after movement is within a predetermined search range and within the first captured image, and if the controller determines that the first area after movement is within the search range but not within the first captured image, the controller may change the shooting direction of the first camera and the second camera using the shooting direction change mechanism and take images again using the first camera and the second camera.

With the above configuration, by taking multiple photographs while changing the camera's shooting direction, it is possible to obtain multiple photographed images of the entire search range. Therefore, the shooting range per photographed image can be made relatively narrow, which increases the resolution of the search area image obtained from the photographed images. Alternatively, it is possible to set the search range wider than the camera's field of view.

2 Terminal building 3 Aircraft 6 Cab 8 Lifting device 11 Passenger boarding bridge 12 Second boarding bridge 20 Detection system 21a First camera 21b Second camera 23 Image processing device 30 Controller 34 Shooting direction change mechanism 36 Height change mechanisms D1, D2 Boarding and disembarking section

Claims (6)

1. A detection system for detecting a landing section of an aircraft, comprising:
a passenger boarding bridge connected to the terminal building;
a first camera and a second camera provided on the passenger boarding bridge;
a height changing mechanism capable of changing the height positions of the first camera and the second camera;
a controller for controlling the operation of the height change mechanism;
an image processing device that detects a boarding and disembarking section of the aircraft from a first photographed image taken by the first camera and a second photographed image taken by the second camera,
the first camera and the second camera are disposed such that a first angle of a first virtual line connecting an optical center of the first camera and a predetermined position on the side of an object photographed by the first camera with respect to a horizontal plane is different from a second angle of a second virtual line connecting an optical center of the second camera and the predetermined position with respect to the horizontal plane;
the image processing device generates a first search area image from the first captured image and generates a second search area image from the second captured image;
the first search area image has a larger area in the vertical direction than the second search area image,
The controller controls the height change mechanism to change the height positions of the first camera and the second camera when the image processing device detects the boarding and alighting area from the first search area image but does not detect the boarding and alighting area from the second search area image. the image processing device divides a first region including a portion of the first captured image to generate the first search area image including the partial region of the first captured image partitioned based on the first region, and divides a second region including a portion of the second captured image to generate the second search area image including the partial region of the second captured image partitioned based on the second region;
the first region and the second region are set to the same region in real space;
The detection system according to claim 1 , wherein the partial area of the first captured image has a larger area in the vertical direction than the partial area of the second captured image. The image processing device includes:
when the boarding and disembarking section is not detected from the first search area image, the first area is moved from one side in the aircraft axis direction of the aircraft on the first photographed image to the other side, thereby sequentially generating the first search area image, and the second area is moved from one side in the aircraft axis direction of the aircraft on the second photographed image to the other side, thereby sequentially generating the second search area image;
The detection system described in claim 2, wherein the search for the boarding and alighting area is repeatedly performed by using a plurality of sequentially generated first search area images while changing the first search area image, and the search for the boarding and alighting area is repeatedly performed by using a plurality of sequentially generated second search area images while changing the second search area image. The detection system described in any one of claims 1 to 3, wherein the first camera is positioned above the second camera. the passenger boarding bridge comprises a cab having a connection portion connected to the boarding and disembarking portion of the aircraft, and a lifting device that raises and lowers the cab;
the first camera and the second camera are provided in the cab,
The detection system according to claim 1 , wherein the height changing mechanism is the lifting device. an imaging direction changing mechanism that changes the imaging directions of the first camera and the second camera by rotating about a predetermined rotation axis that extends in a direction intersecting a horizontal plane;
the image processing device determines whether the first area after movement is within a predetermined search range and within the first captured image;
4. The detection system of claim 3, wherein when the controller determines that the first area after movement is within the search range but not within the first captured image, the controller changes the capturing directions of the first camera and the second camera using the capturing direction change mechanism and captures images again using the first camera and the second camera.