JP7534064B2 - Cargo handling vehicle and control method thereof - Google Patents

Description

The present invention relates to a cargo handling vehicle on which an operator rides and a control method thereof.

In general, a forklift, which is a loading and unloading vehicle, is configured so that an operator rides in it to drive the vehicle and operate the forks. A monitor device is provided in the driver's seat where the operator rides in the forklift, and information about the forklift is displayed on the monitor device (Patent Document 1, etc.). Information about the forklift, such as information about the fork operation, information about the light illumination status, information about the cargo being loaded and unloaded, and information about the brake status, is displayed on the monitor device.

By the way, the operator can get information about the forklift by checking the display on the monitor device, but if other people besides the operator operating the forklift, such as workers working around the forklift or operators of other forklifts, could also get information about the forklift, they would be able to understand the fork operation and avoid danger, or let the operator know if they have forgotten to turn off the lights or apply the brakes.

Japanese Patent Application Publication No. 7-242400

The problem that the present invention aims to solve is to provide a loading vehicle and a control method thereof that can provide information about the loading vehicle to people other than the operator operating the forklift.

To solve the above problems, the loading vehicle of the present invention is a loading vehicle on which an operator rides, and is equipped with a projection unit that projects an image to notify information about the loading vehicle, and a projection determination unit that determines whether the image is to be projected onto the road surface or a wall surface based on the operator's visibility.

The loading vehicle is further equipped with a photographing unit that photographs the road surface and wall surface. It is preferable that the projection determination unit acquires and determines the image photographed by the photographing unit.

Preferably, the projection determination unit includes a collection unit that collects teacher data based on the relationship between color data related to the color of the captured image and illuminance data related to the illuminance, and a visibility score indicating the visibility of the image by the operator. Furthermore, the projection determination unit includes a learning model generation unit that performs machine learning from the teacher data collected by the collection unit and generates and stores a learning model by machine learning, an acquisition unit that acquires current color data and illuminance data at predetermined time intervals, a prediction unit that acquires a visibility score from the learning model by inputting the current color data and illuminance data acquired from the acquisition unit into the learning model generated by the learning model generation unit, and a determination unit that determines whether the image is to be projected onto a road surface or a wall surface based on the visibility score acquired by the prediction unit.

The control method for a loading vehicle according to the present invention is a control method for a loading vehicle on which an operator rides, and the loading vehicle is equipped with a projection unit that projects an image for notifying the operator of information related to the loading vehicle. The control method includes a projection determination step that determines whether the image is to be projected onto a road surface or a wall surface based on the visibility of the operator.

The control method preferably further includes an imaging step of imaging the road surface and the wall surface, and the projection determination step is determined by acquiring the image captured in the imaging step.

Preferably, the control method further includes a collection step of collecting teacher data based on the relationship between color data related to the color and illuminance data related to the illuminance of the captured image and a visibility score indicating the visibility of the image by the operator. The control method also includes a learning model generation step of performing machine learning from the teacher data collected in the collection step and generating and storing a learning model by machine learning, an acquisition step of acquiring current color data and illuminance data at predetermined time intervals, a prediction step of acquiring a visibility score from the learning model by inputting the current color data and illuminance data acquired in the acquisition step into the learning model generated in the learning model generation step, and a decision step of deciding whether the image should be projected onto a road surface or a wall surface based on the visibility score acquired by the prediction step.

The loading vehicle and control method according to the present invention, by being equipped with the above-mentioned configuration, can inform people other than the operator operating the forklift of information related to the loading vehicle, and can clearly project an image expressing information related to the loading vehicle by having the projection determination unit determine whether to project onto the road surface or the wall surface based on the operator's visibility.

FIG. 2 is a side view for explaining a forklift. FIG. 2 is a block diagram for explaining a configuration of a forklift. 10 is a block diagram for explaining a configuration of a projection determination unit. FIG. FIG. 11 is a flowchart illustrating a control method of the projection determination unit. FIG. 11 is a plan view for explaining an image projected from a forklift.

The cargo handling vehicle and its control method according to the present invention will be described below with reference to the drawings.

As shown in FIG. 1, in this embodiment, the cargo handling vehicle 1 is a counterweight type forklift 1. The forklift 1 is configured to be operated by an operator O who rides on it. In this embodiment, the forklift 1 is configured so that the vehicle body can travel and the forks can be raised and lowered by the operator O operating it.

As shown in Figs. 1 and 2, the forklift 1 is equipped with a projection unit 2. The projection unit 2 is configured to project an image 200 for notifying the operator O and people in the vicinity of the forklift 1 of information I.

The information I about the forklift 1 includes, for example, information about the fork operation, information about the light illumination status, information about the cargo being handled, information about the brake status, information about the operation lever status, etc. The forklift 1 is equipped with an information acquisition unit 6 that acquires the information I. The information acquisition unit 6 is configured to acquire these operations and statuses, etc., using the lift device that raises and lowers the forks, the lights, the brakes, the fork load sensors, etc.

As shown in FIG. 5(A), for example, while the forks of forklift 1 are rising, an image 200 indicating that the forks are rising is projected from projection unit 2, and as shown in FIG. 5(B), if the weight of the load lifted by the forks of forklift 1 exceeds the allowable load, an image 200 indicating that the load is overloaded is projected from projection unit 2, so that information I relating to the forklift can be communicated to people other than operator O of forklift 1, such as operator O on board forklift 1, workers working around forklift 1, and operators on board other forklifts.

By recognizing that the forks of the forklift 1 are rising and their current positions, the operator O of the forklift 1 can quickly and appropriately grasp the condition and position of the forks. In addition, by recognizing that the forks of the forklift 1 are rising and their current positions, people other than the operator O of the forklift 1 can be alerted when working or walking.

In this way, by making the operator O of the forklift 1 aware that the load lifted by the forklift 1 is overloaded, the amount of load can be adjusted so that the load is not overloaded. Also, by informing people other than the operator O of the forklift 1 that the forklift 1 is transporting an overloaded load, the other people can warn the operator O or report the situation to a manager or supervisor.

As shown in FIG. 5, the image 200 is projected onto the road surface P in front of and behind the forklift 1, but it may also be projected onto the side of the forklift 1, etc.

As shown in Figures 1 and 2, the forklift 1 is equipped with an imaging unit 3 that captures images of the road surface P and wall surface W. The imaging unit 3 may be, for example, a camera having a CCD sensor, a CMOS sensor, an image processor, or the like, and is configured to acquire captured images. The forklift 1 travels on the horizontal road surface P. The wall surface W is provided in the vertical direction and may include walls within a facility or the outer surfaces of luggage stored on shelves, and may be any vertical surface onto which the image 200 can be projected.

The forklift 1 is equipped with a control unit 5. The control unit 5 is connected to an information acquisition unit 6 and a projection unit 2, and selects an image 200 to be projected from information I acquired by the information acquisition unit 6, and the selected image 200 is projected by the projection unit 2. For example, while the forks are rising and falling, an image 200 representing the forks rising and falling is selected, while the lights have been forgotten to be turned off, an image 200 representing the lights having been forgotten to be turned off is selected, and while an overloaded load is being transported by the forks, an image 200 representing an overload is selected.

The control unit 5 also includes a projection determination unit 4 that determines whether the image 200 is to be projected onto the road surface P or onto the wall surface W. The projection determination unit 4 is configured to determine whether the image 200 is to be projected onto the road surface P or onto the wall surface W based on the visibility of the operator O. This allows the image 200 to be projected clearly.

As shown in FIG. 3, the projection determination unit 4 includes a collection unit 40 that collects teacher data 46. The teacher data 46 includes color data D1 relating to the color of the captured image and illuminance data D2 relating to the illuminance of the captured image. The color data D1 relating to the color of the captured image is the color C1 of the road surface P and the color C2 of the wall surface W. The illuminance data D2 relating to the illuminance of the captured image is the illuminance C3 of the road surface P and the illuminance C4 of the wall surface W.

The projection determination unit 4 includes a learning model generation unit 41 that performs machine learning from the teacher data 46 collected by the collection unit 40 and generates and stores a learning model through machine learning. The learning model generation unit 41 of this embodiment performs supervised learning. In supervised learning, a large amount of teacher data 46, i.e., pairs of input data ID and output data OD, is input to the learning model generation unit 41.

The input data ID includes the color C1 of the road surface P, the color C2 of the wall surface W, the illuminance C3 of the road surface P, and the illuminance C4 of the wall surface W. The output data OD is a visibility score. The input data ID is evaluated as a visibility score indicating whether the image 200 projected on the road surface P or on the wall surface W is more visible to the operator O, and a numerical parameter ranging from 0 to 10 is set.

Cases where the numerical parameter of the visibility score is high, i.e., the visibility of the image 200 by the operator O is judged to be high, include, for example, (1) when a predetermined percentage or more of the color C1 of the road surface P is the same color as the pre-stored color of the road surface P, (2) when the illuminance C3 of the road surface P is equal to or greater than a predetermined illuminance, (3) when a predetermined percentage or more of the color C2 of the wall surface W is the same color as the pre-stored color of the wall surface W, and (4) when the illuminance C4 of the wall surface W is equal to or greater than a predetermined illuminance. The projection determination unit 4 determines whether to project the image 200 onto the road surface P or the wall surface W, based on the above numerical parameters (1) and (2) related to the road surface P and the above numerical parameters (3) and (4) related to the wall surface W.

The visibility score may be set by any one of the numerical parameters: (1) the color C1 of the road surface P, (2) the illuminance C3 of the road surface P, (3) the color C2 of the wall surface W, or (4) the illuminance C4 of the wall surface W, or may be set by a weighted average of the numerical parameters using a weighting coefficient.

In fact, regarding the visibility of the image 200 by the operator O, when most of the colors of the road surface P and the wall surface W are the same as the original colors and the illuminance is high, the operator O can easily see the projected image 200. Therefore, it can be inferred that there is a certain relationship, such as a correlation, between (1) the color C1 of the road surface P, (2) the illuminance C3 of the road surface P, (3) the color C2 of the wall surface W, and (4) the illuminance C4 of the wall surface W and the visibility of the operator O.

The learning model generation unit 41 uses a machine learning algorithm such as a general neural network. The learning model generation unit 41 performs machine learning using correlated input data ID and output data OD as teacher data 46 to generate a model (learning model) that estimates output from input, that is, a model that outputs a fall possibility score when an input data ID is input.

The projection determination unit 4 includes an acquisition unit 45 that acquires the current input data ID at predetermined time intervals. As described above, the input data ID is (1) the color C1 of the road surface P, (2) the illuminance C3 of the road surface P, (3) the color C2 of the wall surface W, and (4) the illuminance C4 of the wall surface W. The acquisition unit 45 is configured to acquire the input data ID based on the image captured by the imaging unit 3. The input data ID is acquired at predetermined time intervals (e.g., one minute).

The projection determination unit 4 includes a prediction unit 42 that predicts whether the image 200 projected onto the road surface P or onto the wall surface W will be more visible to the operator O by applying the learning model generated by the learning model generation unit 41 to the current input data ID acquired from the acquisition unit 45.

The projection determination unit 4 includes a determination unit 43, which determines whether the image 200 is to be projected onto the road surface P or onto the wall surface W based on the output data OD predicted by the prediction unit 42, and outputs a determination signal to the projection unit 2. The projection unit 2 is configured to be rotatable, and is configured to project the image 200 onto the road surface P or onto the wall surface W based on the determination signal from the projection determination unit 4.

As shown in FIG. 4, the projection determination unit 4 executes the following control method. Note that to avoid duplication, parts that have already been explained will be omitted.

The collection unit 40 collects teacher data 46 (collection step: S1). Then, the learning model generation unit 41 performs machine learning from the teacher data 46 collected by the collection unit 40 in the collection step S1, and generates and stores a learning model through machine learning (learning model generation step: S2). The acquisition unit 45 acquires the current input data ID at predetermined time intervals (acquisition step: S3).

The prediction unit 42 predicts the visibility of the image 200 by the operator O by applying the learning model generated in the learning model generation step S2 to the current input data ID acquired in the acquisition step S3 (prediction step: S4). The decision unit 43 decides whether the image 200 is to be projected onto the road surface P or the wall surface W based on the output data OD predicted in the prediction step S4, and outputs a decision signal to the projection unit 2 (decision step: S5).

Although the preferred embodiments of the present invention have been described above, the configuration of the present invention is not limited to these embodiments.
For example, the photographing decision unit 4 may be configured to determine whether the illuminance of the road surface P or the wall surface W is higher based on, for example, the illuminance results from illuminance sensors attached to the road surface P and the wall surface W, rather than on the image captured by the photographing unit 3, and to determine whether the image is to be projected onto the road surface P or the wall surface W.
Furthermore, the cargo handling vehicle 1 is not limited to a counterweight forklift, but may be a reach forklift, a side forklift, an order picking truck, a towing vehicle, or the like.

The effects of this embodiment will be described.
By projecting an image 200 representing information I relating to the loading vehicle 1 onto a road surface P or a wall surface W, the information I relating to the loading vehicle 1 can be communicated to persons other than the operator O of the loading vehicle 1, such as an operator O who is on board and operates the loading vehicle 1, workers working around the loading vehicle 1, and operators O on board other loading vehicles 1.
The projection determination unit 4 determines whether to project the image 200 onto the road surface P or the wall surface W based on the visibility of the operator O, so that the image 200 can be clearly projected. This makes it easier for the operator O who is on board and operating the cargo handling vehicle 1, workers working around the cargo handling vehicle 1, and operators O on other cargo handling vehicles 1 to view the image 200.

1. Forklift (cargo handling vehicle)
Reference Signs List 2 Projection unit 3 Photography unit 4 Projection determination unit 5 Control unit 40 Collection unit 41 Learning model generation unit 42 Prediction unit 43 Determination unit 45 Acquisition unit 46 Teacher data 200 Image P Road surface W Wall surface O Operator ID Input data OD Output data

Claims (2)

A loading vehicle on which an operator rides,
A projection unit that projects an image for notifying information about the loading vehicle;
a projection determination unit that determines whether the image is to be projected onto a road surface or a wall surface based on the visibility of the operator;
An imaging unit that images the road surface and the wall surface,
The projection determination unit is
Acquire an image captured by the imaging unit;
a collection unit that collects teacher data based on a relationship between color data related to a color of the captured image and illuminance data related to an illuminance of the captured image, and a visibility score that indicates visibility of the image by the operator;
a learning model generation unit that performs machine learning on the teacher data collected by the collection unit, and generates and stores a learning model by the machine learning;
an acquisition unit that acquires the current color data and the current illuminance data at predetermined time intervals;
a prediction unit that acquires the visibility score from the learning model by inputting the current color data and the current illuminance data acquired from the acquisition unit into the learning model generated by the learning model generation unit;
A determination unit that determines whether the image is to be projected onto the road surface or onto a wall surface based on the visibility score obtained by the prediction unit. A method for controlling a loading vehicle on which an operator rides, comprising:
The loading vehicle is
a projection unit that projects an image for notifying the operator of information related to the loading vehicle;
The method for controlling the loading vehicle includes:
An imaging step of imaging a road surface and a wall surface;
a projection determination step of acquiring an image captured by the photographing step and determining whether the image is to be projected onto the road surface or the wall surface based on visibility of the operator;
Further, a collection step of collecting teacher data based on a relationship between color data related to the color of the captured image and illuminance data related to the illuminance of the captured image, and a visibility score indicating visibility of the image by the operator;
a learning model generation step of performing machine learning from the teacher data collected in the collection step, and generating and storing a learning model by the machine learning;
an acquisition step of acquiring the current color data and the current illuminance data at predetermined time intervals;
a prediction step of acquiring the visibility score from the learning model by inputting the current color data and the current illuminance data acquired in the acquisition step into the learning model generated in the learning model generation step;
A control method for a loading vehicle, comprising: a determination step of determining whether the image should be projected onto the road surface or onto a wall surface based on the visibility score obtained by the prediction step.

Images