JP7306876B2 - Operation vehicle support system - Google Patents

Description

The present invention relates to a driving support system for work vehicles such as forklifts.

BACKGROUND ART Conventionally, there has been proposed a technology for supporting cargo handling work by taking a picture of the state of cargo (loaded goods) with an in-vehicle camera of a work vehicle such as a forklift and referring to the image (see, for example, Patent Document 1). .

Japanese Patent Application Laid-Open No. 2003-155198

By the way, when transporting a load with a work vehicle such as a forklift, it is necessary to pay close attention to the position of the center of gravity of the load.

That is, for example, when a work vehicle such as a forklift is used to transport a relatively tall (high center of gravity) load, if a sudden operation such as a sharp turn is performed, the center of gravity of the load shifts, causing the load to move. There is a risk of serious accidents such as collapse of cargo and overturning of work vehicles.
However, the conventional technology has a problem that even when the center of gravity of the load becomes high, it is not possible to notify the driver at an appropriate timing.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a driving support system for a work vehicle that can monitor the height of a load and notify the operator at an appropriate timing.

In order to achieve the above object, a driving support system for a work vehicle according to the present invention includes photographing means capable of acquiring an image of a load of the work vehicle, and storage means for storing data of a change state of the image of the load. a determination means for determining the height of the load based on the data of the change state of the image of the load; The gist of the present invention is to provide an informing means for informing an operator of the work vehicle to pay attention to the work vehicle in a case where the work vehicle is in a state of emergency.

Advantageous Effects of Invention According to the present invention, it is possible to provide a driving support system for a work vehicle that can monitor the height of a load and notify it at an appropriate timing.

1 is a side view showing a forklift to which a driving support system for work vehicles according to an embodiment is applied; FIG. 1 is a functional block diagram showing the functional configuration of a driving support system for a work vehicle according to an embodiment; FIG. FIG. 2 is a block diagram showing a configuration example of determination means forming part of the driving support system for the work vehicle according to the embodiment; FIG. 4 is a screen diagram showing an example of a no-load state by an in-vehicle camera of a forklift to which the driving support system for a work vehicle according to the embodiment is applied; FIG. 10 is a screen diagram showing an example of a pallet or a low-height load by an in-vehicle camera of a forklift to which the driving support system for a work vehicle according to the embodiment is applied; FIG. 10 is a screen diagram showing an example of a somewhat tall load by an in-vehicle camera of a forklift to which the driving support system for a work vehicle according to the embodiment is applied; FIG. 4 is a screen diagram showing an example of a tall load (or lifted up) by an in-vehicle camera of a forklift to which the driving support system for work vehicles according to the embodiment is applied; 1 is a block diagram showing a configuration example of a digital tachograph applied to a driving support system for a work vehicle according to an embodiment; FIG. FIG. 4 is an explanatory diagram showing a display example of a digital tachograph applied to the driving support system for the work vehicle according to the embodiment; 4 is a flow chart showing a procedure of notification processing executed by the driving support system for the work vehicle according to the embodiment; 7 is a flow chart showing the continuation of the processing procedure of the notification process executed by the driving support system for the work vehicle according to the embodiment;

A driving support system S1 for a work vehicle according to an embodiment of the present invention will be described with reference to FIGS. 1 to 9. FIG.
(Configuration example of work vehicle)
Referring to FIG. 1, a forklift F will be described as a configuration example of a work vehicle to which driving support system S1 for a work vehicle according to the present embodiment can be applied.

A forklift F as a work vehicle includes a fork portion 103 that moves up and down, driving wheels (rear wheels) 104, traveling wheels (front wheels) 105, an instrument panel 116, a steering wheel 110, a shift lever 111, and the like.

The forklift F may be a so-called engine vehicle using an internal combustion engine such as a gasoline engine as a drive source 102, or an electric vehicle driven by electric power supplied from a rechargeable secondary battery (battery) or the like from a generator or the like. A so-called battery car using a motor as the drive source 102 may be used.

Above the mast portion 115 that supports the fork portion 103, there is a photographing means (in-vehicle camera) 101 capable of acquiring an image of the load P (for example, pallets P1 to P3) and a background image of the load P (P1 to P3). is provided. The photographing range 500 of the in-vehicle camera 101 is set to a range in which the fork portion 103, the load P (P1 to P3), etc. can be photographed. Further, the location where the in-vehicle camera 101 is provided is not limited to the mast portion 115 as long as the image of the fork portion 103 can be acquired.

In the instrument panel 116, a digital tachograph DT1 and the like are provided as vehicle-mounted equipment that constitutes a part of the driving support system S1. Details of the digital tachograph DT1 and the like will be described later.
An acceleration sensor (G sensor) 201 is attached to the vehicle body of the forklift F to detect acceleration (longitudinal G, lateral G, and vertical G) during movement.

An operator (driver) of the forklift F operates the steering wheel 110, the speed change lever 111, and pedals such as an accelerator pedal and a brake pedal (not shown) to raise and lower the fork portion 103, move the forklift F forward, backward, turn right, Make the user perform an action such as turning left.

The driving support system S1 for a work vehicle according to the present embodiment monitors the height of the loads P (P1 to P3) based on the information captured by the vehicle-mounted camera 101, and detects the height of the loads P (P1 to P3). When the load exceeds a preset threshold value, a warning is given on the screen or by voice, thereby preventing collapse of cargo and overturning of the forklift F itself.
Note that the working vehicle is not limited to the forklift F, and may be a transport vehicle or the like as long as it is a vehicle that loads, unloads, transports, etc. cargo, products, and the like in the premises of a factory, warehouse, or the like.

(Function configuration of driving support system for work vehicle)
2 to 7, the functional configuration and the like of the driving support system S1 for the work vehicle according to the present embodiment will be described.

Here, FIG. 2 is a functional block diagram showing the functional configuration of the driving support system S1 for work vehicles according to the embodiment, and FIG. A block diagram showing an example, FIG. 4 is a screen diagram showing an example of no load state by an in-vehicle camera of a forklift F to which the driving support system S1 for work vehicles is applied, and FIG. 5 is a pallet P1 or a low-height load FIG. 6 is a screen diagram showing an example of a relatively tall load P2, and FIG. 7 is a screen diagram showing an example of a tall load P3 (or in a lifted state). is.

First, as shown in FIG. 2, the driving support system S1 for the work vehicle includes an in-vehicle camera or the like capable of acquiring an image of the load P (P1 to P3) loaded on the fork block 103 of the forklift F as the work vehicle. A photographing means 101 configured, a storage means 301 configured by a flash memory or the like for storing data of the image change state of the load P (P1 to P3), and an image change of the load P (P1 to P3). Determination means 302 comprising a CPU or the like for determining the height of the load P (P1 to P3) based on the state data, and the height of the load P (P1 to P3) determined by the determination means 302 and an informing means (speaker 404, display 405) for informing the operator of the forklift F to pay attention when the forklift F exceeds a preset threshold value.

Further, the driving support system S1 includes a clock means 303 including a timer or the like, and an acceleration sensor (G sensor) 201 for detecting the acceleration of the forklift F.

As shown in FIG. 3 and the like, the determination unit 302 sets a plurality of initial detection frames a1 and a2 (see FIG. 4) at positions where the load P (for example, pallets P1 to P3) is assumed to be loaded. Frame setting means 601 and travel determination means 602 for determining whether or not the work vehicle (forklift F) is running based on the amount of movement of an image within the initial detection frame 601 (for example, the image of the pallet P1 in FIG. 5). and load determination means 603 for determining the presence or absence of the load P (P1 to P3) based on whether or not the feature points of the image within the initial detection frames a1 and a2 have moved.
Note that the initial detection frame setting means 601, travel determination means 602, and load determination means 603 can be configured by software or the like that can be executed by a CPU or the like.
In FIG. 4 and the like, detection frames c1 and c2 arranged on the left and right sides of the screen G1 are frames for detecting the moving direction and turning direction of the work vehicle (forklift F).

Further, the determination unit 302 detects an additional object above the image when the load determination unit 603 determines that there is a load P (P1 to P3) while the travel determination unit 602 determines that the vehicle is traveling. An additional detection frame setting means 604 for setting a frame b1 (see FIG. 5, etc.) is provided.

Then, the additional detection frame setting means 604 adds a load (for example, as shown in FIG. 6, the palette P1 In the case where another pallet P2 is added on top of Add and set above.

Based on the position of the highest additional detection frame (additional detection frame b2 in FIG. 6, additional detection frame b3 in FIG. 7, etc.) set by the additional detection frame setting means 604, the determination means 302 determines the position of the entire load. Height can be determined.

Furthermore, the determination means 302 determines an additional detection frame (for example, additional detection frame b1) other than the highest additional detection frame (for example, additional detection frame b2, b3, etc.) set by the additional detection frame setting means 604, and a predetermined It is possible to control not to process the image in the initial detection frame (for example, initial detection frame a1, a2, etc.). As a result, the processing load can be reduced, and processing such as height determination can be performed at a higher speed.

When the determination result confirms that there is a load P, as shown in FIG. may

Threshold changing means for changing a turning warning threshold for determining whether or not to issue a warning based on the lateral G of the work vehicle (forklift F) according to the height of the load P1 or the like determined by the determining means 302. 605 may be further provided. As a result, when the load P1 or the like is relatively high and the center of gravity is on the upper side, and the collapse of the load or the overturn of the forklift F itself is likely to occur, the turning alarm threshold value can be set and changed strictly. Conversely, when the load P1, etc. is relatively low and the center of gravity is on the lower side, and collapse of the load and overturning of the forklift F itself are unlikely to occur, the turning alarm threshold can be set loosely.

Furthermore, as shown in FIG. 2, by providing an acceleration sensor 201 for detecting the acceleration of the work vehicle (forklift F), the notification means (speaker 404, display 405) can detect the value detected by the acceleration sensor 201 in the vertical direction. When the predetermined value is exceeded, there is a possibility that the road surface may be uneven, so a notification may be issued to prompt the work vehicle (forklift F) to slow down. As a result, the operator can reduce the speed of the forklift F based on the notification, thereby suppressing the shaking of the load P1 and the like.

Further, the determination unit 302 determines whether the loaded object is detected based on the amount of movement of the feature points of the image within the highest additional detection frame set by the additional detection frame setting unit 604 (for example, the additional detection frame b3 in FIG. 7). The amplitude of the swing of P3 is detected, and if the amplitude exceeds a predetermined value, the notification means (speaker 404, display 405) prompts the work vehicle to slow down until the amplitude falls below a predetermined value. Notification can be made. As a result, the operator can decelerate the forklift F until the amplitude is sufficiently reduced based on the notification, thereby suppressing the swinging of the load P1 and the like.

(Configuration example of a digital tachograph)
The configuration and the like of the digital tachograph DT1 will be described with reference to FIGS. 8 and 9. FIG.
Here, FIG. 8 is a block diagram showing a configuration example of a digital tachograph DT1 as a vehicle-mounted device applied to the driving support system S1 of the work vehicle, and FIG. 9 is a digital tachograph applied to the driving support system S1 of the work vehicle. FIG. 4 is an explanatory diagram showing a display example in DT;

A digital tachograph DT1 as an on-vehicle device shown in FIG. It is recorded every time.

Digital tachograph DT1 includes CPU 11, speed/engine speed I/F 12, external input I/F 13, sensor input I/F 14, GPS receiver 202, CAN_I/F 16, recording means 301, card I/F 18, voice I/F 19. , speaker 204, RTC (clock IC) 21, delivery/receipt button SW input section 22, display controller 23, communication section 24, power supply section 25, memory 26, LED display section 403, land transportation ON/OFF button SW input section 31. have.

The CPU 11 is a control unit composed of a microcomputer and controls the entire digital tachograph DT1 according to a program. Programs executed by the CPU 11 and data such as constants necessary for control are held in the memory 26 . The memory 26 is non-volatile, and predetermined programs and data can be read and written.
The memory 26 stores two threshold tables 26a and 26b and an area information table 26c as constant data.
The recording means 301 is a non-volatile memory from which data can be read and written, and is used for recording and holding operating data.

A card I/F (interface) 18 is an interface for connecting a memory card conforming to a predetermined standard to the CPU 11, and has a card slot for detachably holding the memory card. A predetermined memory card 55 possessed by each operator (operator) who drives the forklift (work vehicle) F is attached to the card I/F 18 . This memory card 55 incorporates a non-volatile memory and is used in a state in which a number or the like for identifying each operator is registered.

The audio I/F 19 is an interface for synthesizing and outputting pseudo-audio signals, and can output audio signals corresponding to various messages under the control of the CPU 11 . A speaker 404 as a notification means is connected to the output of the audio I/F 19 .

An RTC (real time clock) 21 keeps track of current time information by constantly counting predetermined clock pulses. The RTC 21 also serves as a timer (clock means) 303 . By controlling the RTC 21 , the CPU 11 can obtain current time information and use the timer (clocking means) 303 .

An ON/OFF signal for the leaving/receiving button is input to the leaving/receiving button SW input unit 22 . Therefore, the CPU 11 monitors the ON/OFF state of the leaving/receiving button via the leaving/receiving button SW input unit 22, and can grasp whether the own vehicle is in the leaving/receiving state. ) F's working hours can be obtained.

The display controller 23 controls display contents of the display 405 . The display 405 is composed of, for example, a liquid crystal display device, an organic EL display device, or the like, and can display various information in combination with graphic display.

The LED display unit 403 incorporates a plurality of LEDs (light emitting diodes), and can display the state of communication and operation by lighting, extinguishing, or blinking each LED under the control of the CPU 11 .
A land transportation ON/OFF button SW input unit 31 is operated by an operator or the like, and is used to manually switch the land transportation mode on and off.

A speed pulse and a rotation pulse are input to the speed/engine speed I/F 12 from a vehicle speed sensor and an engine speed sensor, respectively. The speed/engine speed I/F 12 converts the input signal into a signal suitable for processing by the CPU 11 .
The external input I/F 13 is an interface for connecting an external device to the CPU 11, and is connected to an in-vehicle camera (photographing means) 101 and the like.

A sensor input I/F 14 is an interface for connecting various sensors to the CPU 11 . Signals from a temperature sensor that detects engine temperature, a fuel sensor that detects fuel amount, a G sensor 201 that detects vehicle acceleration (G value), and the like are input to the sensor input I/F 14 .

A GPS receiver (GPS sensor) 202 receives radio waves from a plurality of GPS (Global Positioning System) satellites with a GPS antenna 202a, performs predetermined calculation processing, and obtains the current position of the forklift (working vehicle) F from the received signals. Get the latitude and longitude information to represent. Thereby, the position information of the forklift (work vehicle) F can be acquired.

CAN_I/F 16 is an interface for connecting to a CAN (Controller Area Network) standard communication network. The CPU 11 can communicate with various devices connected to the communication network on the vehicle via the CAN_I/F 16 . In practice, communication of various data such as speed, engine speed, and fuel amount is performed.
The communication unit 24 incorporates a predetermined wireless communication interface, and can perform data communication such as operation data with the outside of the vehicle using a wireless communication line.

The power supply unit 25 generates a stable voltage (for example, +5 [V]) necessary for the operation of the CPU 11 and the like based on the power supplied from the vehicle battery or the like, and the digital tachograph DT1 operates when the ignition switch is on. Power is supplied to each circuit.

Then, the determination means 302 composed of the CPU 11 or the like monitors the height of the load P (P1 to P3) based on the information captured by the vehicle-mounted camera 101, and determines whether the height of the load P (P1 to P3) is When the preset threshold value is exceeded, as shown in FIG. Notify.
As a result, the operator can decelerate based on the notification, thereby preventing collapse of cargo and overturning of the forklift F itself.

(Regarding notification processing)
Next, a processing procedure of notification processing executed by the driving support system S1 for the work vehicle will be described with reference to the flowcharts of FIGS. 10A and 10B.
First, in step S10, it is determined whether or not the forklift (work vehicle) F has left the garage.

The criterion for determining whether or not the forklift (working vehicle) F has left the warehouse is whether or not it has moved a predetermined distance from a predetermined parking position based on the position information acquired by the position information acquisition means (GPS sensor) 202, for example. etc. can be determined.
In step S11, photographing by the in-vehicle camera (photographing means) 101 and recording in the storage means 301 are started, and the process proceeds to step S12.

In step S12, as shown in FIG. 4 and the like, it is determined whether or not the images within the detection frames c1 and c2 set on the screen G1 are moving. In that case, the process proceeds to step S13.
In step S13, it is determined that the forklift (work vehicle) F is running, and the process proceeds to step S14.

In step S14, it is determined whether or not the feature points within the initial detection frames a1 and a2 shown in FIG. 5 and the like have moved. The process proceeds to step S15.
In step S15, it is determined that there is a load P on the fork portion 103, and the process proceeds to step S16.

In step S16, as shown in FIGS. 5 to 7, additional detection frames b1 to bn (n is an integer of 2 or more) are set. Note that the additional detection frame may be set each time the load P increases.

Next, in step S17, based on the position of the highest additional detection frame (for example, the additional detection frame b2 in FIG. 6 or the additional detection frame b3 in FIG. 3), the load (for example, the load P2 in FIG. 6) is detected. , or the height of the load P3 in FIG. 7) is determined, and the process proceeds to step S18.

In step S18, it is determined whether or not the height of the load exceeds a preset threshold.

In step S19, a message such as "Please slow down" is displayed on the display 405, and the speaker 404 also issues a voice notification, and the process ends.

As described above, according to the driving support system for the work vehicle according to the present embodiment, the operator (operator) decelerates based on the notification, thereby preventing collapse of cargo and overturning of the forklift F itself. can do.

Although the driving support system for a work vehicle according to the present invention has been described above based on the illustrated embodiments, the present invention is not limited to this, and the configuration of each part may be any configuration having similar functions. can be replaced with

S1: Driving support system for work vehicle F: Forklift (work vehicle)
DT1: Digital tachograph (in-vehicle device)
P (P1 to P3) … Load (Pallet)
101... Imaging means (in-vehicle camera)
102... Driving means 103... Fork portion 201... Acceleration sensor (G sensor)
301... Storage means 302... Determination means 404... Speaker (notification means)
405... Indicator (informing means)
601 Initial detection frame setting means 602 Travel determination means 603 Loaded object determination means 604 Additional detection frame setting means 605 Threshold changing means a1, a2 Initial detection frames b1 to b3 Additional detection frames

Claims (5)

a photographing means capable of acquiring an image of the load of the work vehicle and a background image ;
a storage means for storing data of the image of the load and the background image acquired by the photographing means as image data ;
initial detection frame setting means for setting a plurality of initial detection frames at positions where the loading of the load is estimated with respect to the image data;
travel determination means for determining whether or not the work vehicle is running based on the amount of movement of the image data within the initial detection frame;
load determining means for determining the presence or absence of the load based on whether or not the feature point indicated by the image data within the initial detection frame has moved along with the loading of the load;
an additional detection frame setting means for setting an additional detection frame above the image data when the load determination means determines that the load is present while the travel determination means determines that the vehicle is traveling; ,
determination means for determining the height of the entire load based on the position of the uppermost additional detection frame set by the additional detection frame setting means;
a notification means for notifying an operator of the work vehicle to pay attention when the height of the load determined by the determination means exceeds a preset threshold ;
The additional detection frame setting means determines whether or not the load has been added based on whether or not the feature point indicated in the image data within the additional detection frame has moved due to the addition of the load. and setting a new addition detection frame above the previous addition detection frame each time it is determined that the load has been added. The determination means is
2. The operation according to claim 1 , wherein the processing of the image data in the additional detection frame other than the highest additional detection frame set by the additional detection frame setting means and in the predetermined initial detection frame is not performed. Vehicle driving assistance system. It is characterized by further comprising threshold changing means for changing a turning warning threshold for determining whether or not to issue a warning based on the lateral G of the work vehicle according to the height of the load determined by the determining means. 3. The driving support system for a work vehicle according to claim 1 or 2 . further comprising an acceleration sensor that detects acceleration of the work vehicle;
4. The informing device according to any one of claims 1 to 3 , wherein when the value detected by the acceleration sensor in the vertical direction exceeds a predetermined value, the informing means issues a notification to urge the vehicle to reduce its speed. 1. A driving support system for a work vehicle according to claim 1. The determination means is
detecting the amplitude of the swing of the load based on the amount of movement of the feature points shown in the image data within the uppermost additional detection frame set by the additional detection frame setting means;
When the amplitude exceeds a predetermined value, the notification means issues a notification urging the work vehicle to slow down until the amplitude becomes equal to or less than a predetermined value. Item 5. The work vehicle driving support system according to any one of Item 4 .

Images