HK1238381A1 - Ar technology-based artificial auxiliary method, device, system and host computer - Google Patents

Description

Artificial assistance method, device and system based on AR technology and upper computer

Technical Field

The invention relates to the field of automatic warehousing, in particular to an artificial assistance method, device and system based on AR technology and an upper computer.

Background

With the application of a large number of AS/RS (automatic Storage and Retrieval System) automatic three-dimensional libraries, commodities are stored on a goods shelf by a stacker with a tray AS a Storage unit when being warehoused. There is a certain demand for the pallet at the warehousing end. At present, a more mature stacking scheme is that stacking is carried out through a robot, the robot is limited by the fact that the robot is expensive and not used in a large amount, the robot possibly cannot meet requirements for large-amount warehousing in a sales promotion period, manual participation in stacking can be generated as supplement, and the characteristic of manual stacking is that the stacking type of stacking is not standard and is not controllable.

Disclosure of Invention

The invention aims to solve the technical problem of providing an artificial assistance method, a device and a system based on AR technology and solving the problem of inaccurate stacking position by an upper computer.

According to one aspect of the present invention, an artificial assistance method based on an augmented reality AR technology is provided, including: the AR imaging equipment sends the acquired tray image information to an upper computer; the upper computer generates a superposed picture of the articles at the stacking position of the tray according to the stacking position of the articles on the tray; the AR imaging equipment presents the superposed pictures of the objects, which are sent by the upper computer, at the stacking position of the tray to a user, so that the user can stack the objects at the stacking position of the tray according to the superposed pictures presented by the AR imaging equipment.

Further, the method further comprises: the AR imaging equipment sends position information of the AR imaging equipment to the upper computer; and the upper computer evaluates the posture of the AR imaging equipment according to the position information of the AR imaging equipment and the tray image information and determines the orientation direction of the AR imaging equipment relative to the tray.

Further, the upper computer generates a superimposed image of the article located at the stacking position of the tray according to the stacking position of the article on the tray, and the superimposed image includes: the upper computer receives the article bar code sent by the bar code scanning equipment BCR, identifies the article according to the article bar code and generates a virtual image of the article; and rendering the virtual image of the article to the stacking position of the tray to form a superposed picture according to the orientation direction of the AR imaging device relative to the tray and the stacking position of the article on the tray.

Further, the method further comprises: the upper computer receives the counting number of the article sent by the BCR and the size information of the article sent by the volume measuring device; and determining the stacking position of the articles on the tray according to the image information of the tray and the counting number and the size information of the articles.

Further, the method further comprises: AR imaging equipment receives pile up neatly upper limit information that the host computer sent to remind the user to end pile up neatly article.

According to another aspect of the present invention, there is also provided an artificial assistance device based on augmented reality AR technology, including: the tray image sending unit is used for sending the acquired tray image information to the upper computer so that the upper computer can generate a superposed picture of the articles positioned at the stacking position of the tray according to the stacking position of the articles on the tray; the stacking drawing receiving unit is used for receiving stacking pictures of the articles positioned at the stacking position of the tray, which are sent by the upper computer; and the superposed picture presenting unit is used for presenting the superposed picture to a user so that the user can stack the objects at the stacking position of the tray according to the superposed picture presented by the AR imaging equipment.

Further, the device also comprises a position information sending unit; the position information sending unit sends the position information of the AR imaging device to the upper computer, so that the upper computer carries out posture assessment on the AR imaging device according to the position information of the AR imaging device and the tray image information, the orientation direction of the AR imaging device relative to the tray is determined, and the virtual image of the object is rendered to the stacking position of the tray to form a superimposed picture according to the orientation direction of the AR imaging device relative to the tray and the stacking position of the object on the tray.

Further, the device also comprises an information reminding unit; the information reminding unit is used for receiving stacking upper limit information sent by the upper computer so as to remind a user of finishing stacking objects.

According to another aspect of the present invention, there is also provided a host computer, including: the tray image receiving unit is used for receiving tray image information acquired by the AR imaging equipment; the stacking image generating unit is used for generating a stacking image of the article at the stacking position of the tray according to the stacking position of the article on the tray; and the superposed picture sending unit is used for sending the superposed pictures of the objects at the stacking position of the tray to the AR imaging equipment so that the user can stack the objects at the stacking position of the tray through the superposed pictures presented by the AR imaging equipment.

Further, the upper computer also comprises a position information receiving unit and a posture evaluating unit; the position information receiving unit is used for receiving the position information of the AR imaging equipment sent by the AR imaging equipment; the posture evaluation unit is used for carrying out posture evaluation on the AR imaging device according to the position information of the AR imaging device and the tray image information and determining the orientation direction of the AR imaging device relative to the tray.

Furthermore, the upper computer also comprises a bar code scanning device BCR information receiving unit and a virtual image generating unit; the BCR information receiving unit is used for receiving the article bar code sent by the BCR; the virtual image generating unit is used for identifying the article according to the article bar code and generating a virtual image of the article; the superposed picture generating unit is used for rendering the virtual image of the article to the stacking position of the tray to form a superposed picture according to the orientation direction of the AR imaging device relative to the tray and the stacking position of the article on the tray.

Further, the upper computer also comprises an article volume receiving unit and a stacking position determining unit; the BCR information receiving unit is used for receiving the counting number of the article sent by the BCR; the article volume receiving unit is used for receiving the dimension information of the article sent by the volume measuring device; the stacking position determining unit is used for determining the stacking position of the articles on the tray according to the image information of the tray, the counting number and the size information of the articles.

Further, the upper computer further comprises an upper limit information sending unit; the upper limit information sending unit is used for sending stacking upper limit information to the AR imaging equipment so as to remind a user of finishing stacking objects through the AR imaging equipment.

According to another aspect of the invention, an artificial assistance system based on Augmented Reality (AR) technology is further provided, and the artificial assistance system comprises the artificial assistance device based on AR technology and the upper computer.

According to another aspect of the present invention, an artificial assistance system based on augmented reality AR technology is also provided, including: a memory; and a processor coupled to the memory, the processor configured to perform the method as described above based on instructions stored in the memory.

According to another aspect of the present invention, a computer-readable storage medium is also proposed, on which computer program instructions are stored, which instructions, when executed by a processor, implement the steps of the above-described method.

Compared with the prior art, the AR imaging equipment sends the acquired tray image information to the upper computer, so that the upper computer can generate a superposed picture of the article positioned at the stacking position of the tray according to the stacking position of the article on the tray; AR imaging equipment presents the stack picture that the article that the host computer sent is located the pile-up position of tray for the user to the user is in the pile-up position pile up neatly article of tray, makes the article pile up neatly more neat, can solve the inaccurate problem in pile up neatly position.

Other features of the present invention and advantages thereof will become apparent from the following detailed description of exemplary embodiments thereof, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description, serve to explain the principles of the invention.

The invention will be more clearly understood from the following detailed description, taken with reference to the accompanying drawings, in which:

fig. 1 is a schematic flow chart of an embodiment of the artificial assistant method based on AR technology according to the present invention.

Fig. 2 is a schematic flow chart of another embodiment of the artificial assistant method based on AR technology according to the present invention.

FIG. 3 is a flowchart illustrating a manual assistance method based on AR technique according to another embodiment of the present invention.

Fig. 4 is a schematic diagram of an application scenario of the present invention.

Fig. 5A is a schematic view of a virtual image of an article on a stacking position of a tray, which is seen by a user through AR glasses.

Fig. 5B is another schematic view of the virtual image of the article on the stacking position of the tray as seen by the user through the AR glasses.

FIG. 6 is a schematic structural diagram of an artificial assistant device based on AR technology according to an embodiment of the present invention.

FIG. 7 is a schematic structural diagram of another embodiment of the artificial assistant device based on AR technology according to the present invention.

Fig. 8 is a schematic structural diagram of an embodiment of the upper computer of the present invention.

Fig. 9 is a schematic structural diagram of another embodiment of the upper computer of the present invention.

FIG. 10 is a schematic structural diagram of an embodiment of the artificial assistance system based on AR technology according to the present invention.

FIG. 11 is a schematic structural diagram of an artificial assistant system based on AR technology according to still another embodiment of the present invention.

FIG. 12 is a schematic structural diagram of an artificial assistant system based on AR technology according to another embodiment of the present invention.

Detailed Description

Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.

Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.

In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to specific embodiments and the accompanying drawings.

Fig. 1 is a schematic flow chart of an embodiment of an artificial assistance method based on AR (Augmented Reality) technology according to the present invention. The method comprises the following steps:

in step 110, the AR imaging device sends the acquired tray image information to the upper computer. The AR imaging device can be AR glasses or other screen devices capable of realizing AR imaging. The AR imaging device can be provided with a camera, the camera can acquire image information of the tray in real time, and the AR imaging device sends the acquired image information of the tray to the upper computer.

In step 120, the upper computer generates a superimposed image of the article positioned at the stacking position of the tray according to the stacking position of the article on the tray. The upper computer can determine the stacking position of the article on the tray according to the size information of the article and the size information of the tray, and then generates a superposition picture of the article at the stacking position of the tray by utilizing the AR technology. The AR technology is a technology of calculating the position and angle of a camera image in real time and adding corresponding images, videos, and 3D models, and the objective of the technology is to fit a virtual world on a screen over a real world and perform interaction.

In step 130, the AR imaging device presents the superimposed image of the object located at the stacking position of the tray sent by the upper computer to the user, so that the user stacks the object at the stacking position of the tray according to the superimposed image presented by the AR imaging device. The virtual images of the articles are seen at the relative positions of the trays by the AR imaging equipment worn by the user, and then the articles can be stacked by the user according to the virtual images of the articles at the relative positions of the trays, so that the articles are stacked orderly.

In the embodiment, the AR imaging equipment sends the acquired tray image information to the upper computer, so that the upper computer can generate a superposed picture of the articles at the stacking position of the tray according to the stacking position of the articles on the tray; AR imaging equipment presents the stack picture that the article that the host computer sent is located the pile-up position of tray for the user to the user is in the pile-up position pile up neatly article of tray, makes the article pile up neatly more neat, can solve the inaccurate problem in pile up neatly position.

Fig. 2 is a schematic flow chart of another embodiment of the artificial assistant method based on AR technology according to the present invention. The method comprises the following steps:

in step 210, the AR imaging device sends the acquired tray image information to the upper computer. Wherein, can paste the sign on the tray, the sign on the AR imaging equipment passes through the tray is discerned to the camera.

In step 220, the AR imaging device sends the position information of the AR imaging device to the upper computer. Because the AR imaging equipment is provided with the gyroscope sensor, the gyroscope sensor can measure the position and the state information of the AR imaging equipment.

In step 230, the upper computer performs attitude evaluation on the AR imaging device according to the position information of the AR imaging device and the tray image information, and determines the azimuth direction of the AR imaging device relative to the tray. The upper computer can process the color threshold value of the image information of the tray, extract a rough contour line and extract a contour line which is a quadrangle by polygon approximation.

In step 240, the upper computer receives the article barcode sent by the Bar Code Reader (BCR), identifies the article according to the article barcode, and generates a virtual image of the article.

In step 250, the upper computer renders the virtual image of the article to the stacking position of the tray to form a superimposed picture according to the orientation of the AR imaging device relative to the tray and the stacking position of the article on the tray. Wherein, the host computer can confirm the position of putting things in good order in the tray according to the count number of article, size information and the size information of tray.

In step 260, the upper computer sends the overlay image of the article located at the stacking position of the tray to the AR imaging device.

At step 270, the AR imaging device presents the overlay to the user so that the user palletizes the items in the palletized position of the pallet.

In this embodiment, the host computer receives tray image information and positional information that AR imaging device sent to carry out the gesture aassessment to AR imaging device, confirm AR imaging device and for the azimuth of tray directional, the virtual image of article that will generate is played up and is formed the stack on the pile position of tray, and send to AR imaging device so that the stack that the user appears through AR imaging device pile up neatly the article at the pile position of tray position, consequently can solve artifical pile up neatly and the uncontrollable condition.

FIG. 3 is a flowchart illustrating a manual assistance method based on AR technique according to another embodiment of the present invention. The method comprises the following steps:

at step 310, goods to be palletized may be delivered to a palletizing location via a conveyor line, as illustrated in FIG. 4.

In step 320, the commodity is identified by the BCR scanning system above the conveying line, the commodity is counted, and the commodity bar code and the counting code are sent to the upper computer.

In step 330, the volume measuring device scans the size of the commodity, i.e. the length, the width and the height, and sends the commodity size information to the upper computer.

In step 340, after the user wears the AR glasses, the image information of the tray and the position information of the AR glasses are transmitted to the upper computer through the AR glasses.

In step 350, the upper computer determines the stacking position of the goods on the tray according to the image information of the tray and the counting number and the size information of the goods.

In step 360, the upper computer performs attitude evaluation on the AR glasses according to the position information of the AR glasses and the tray image information, and determines the orientation of the AR glasses relative to the tray.

In step 360, the upper computer identifies the commodity according to the commodity bar code to generate a virtual image of the commodity; and rendering the virtual image of the commodity to the stacking position of the tray to form a superposed picture according to the orientation of the AR glasses relative to the tray and the stacking position of the commodity on the tray.

In step 370, the upper computer sends the superimposed image of the commodity located at the stacking position of the tray to the AR glasses.

In step 380, the user sees the generated virtual images of the commodities at the stacking positions of the trays through the AR glasses, and the commodities are stacked according to the virtual images. As shown in fig. 5A and 5B, wherein the arrow in fig. 5B indicates a virtual image of the ith product at the tray stacking position.

In step 390, the stack is sent to the upper limit of the tray, and the upper computer sends stack upper limit information to the AR glasses to remind the user to finish stacking the commodities.

Wherein, can set up the upper limit of putting things in good order of every kind of tray in the host computer, the setting of upper limit is based on two aspects of height and weight. For example, the requirements of the equipment in the current storehouse on pallet goods are that the height is less than or equal to 1.5 meters, and the weight is less than or equal to 300 kilograms, so that the upper computer can simultaneously calculate two conditions when the goods are stacked, and any one condition is met first, namely the upper limit of stacking is reached. Then, the operator is informed of reaching the upper stacking limit through AR glasses, and stacking is stopped.

In the above embodiment, the user can see the virtual images of the commodities at the stacking positions of the trays by using the AR glasses, and further can stack the commodities according to the virtual images, so that the commodities can be stacked orderly. In addition, as the commodities are stacked neatly, when the robot unstacks or picks the commodities, the robot can unstack according to the stacking standard, and the operation cost is reduced.

FIG. 6 is a schematic structural diagram of an artificial assistant device based on AR technology according to an embodiment of the present invention. The device can be an AR glasses or other screen devices capable of realizing AR imaging, wherein the device includes a tray image sending unit 610, a superimposed picture receiving unit 620 and a superimposed picture presenting unit 630, wherein:

the tray image sending unit 610 is configured to send the acquired tray image information to the upper computer, so that the upper computer generates a superimposed picture of the stacking position of the article on the tray according to the stacking position of the article on the tray. The upper computer can determine the stacking position of the article on the tray according to the size information of the article and the size information of the tray, and then generates a superposition picture of the article at the stacking position of the tray by utilizing the AR technology. The overlay image receiving unit 620 is configured to receive an overlay image of the stacking position of the tray where the article is sent by the upper computer. The overlay screen presenting unit 630 is used to present an overlay screen to the user so that the user stacks the articles at the stacking position of the tray. The virtual images of the articles are seen at the relative positions of the trays by the AR imaging equipment worn by the user, and then the articles can be stacked by the user according to the virtual images of the articles at the relative positions of the trays, so that the articles are stacked orderly.

In the embodiment, the device sends the acquired tray image information to the upper computer, so that the upper computer can generate a superposed picture of the articles at the stacking position of the tray according to the stacking position of the articles on the tray; and the superposed pictures of the articles sent by the upper computer at the stacking positions of the trays are displayed to a user, so that the user can conveniently stack the articles at the stacking positions of the trays, the articles are stacked more orderly, and the problem of inaccurate stacking position can be solved.

FIG. 7 is a schematic structural diagram of another embodiment of the artificial assistant device based on AR technology according to the present invention. The apparatus includes a tray image transmitting unit 710, a position information transmitting unit 720, a superimposed picture receiving unit 730, and a superimposed picture presenting unit 740, wherein:

the tray image sending unit 710 is configured to send the acquired tray image information to an upper computer. The position information sending unit 720 is used for sending the position information of the AR imaging device to the upper computer. And the upper computer performs attitude evaluation on the AR imaging equipment according to the position information of the AR imaging equipment and the tray image information, and determines the orientation and the direction of the AR imaging equipment relative to the tray. In addition, the upper computer identifies the article according to the article bar code, generates a virtual image of the article, and renders the virtual image of the article to the stacking position of the tray according to the orientation of the AR imaging device relative to the tray and the stacking position of the article on the tray to form a superposition picture. The overlay image receiving unit 730 is configured to receive an overlay image of the stacking position of the tray where the article is sent by the upper computer. The overlay screen presenting unit 740 is configured to present an overlay screen to a user so that the user stacks the articles at the stacking position of the tray.

In the embodiment, the device sends the acquired tray image information and the position information of the device to the upper computer, so that the upper computer can evaluate the posture of the AR imaging equipment according to the position information of the AR imaging equipment and the tray image information, determine the orientation direction of the AR imaging equipment relative to the tray, and render the virtual image of the article to the stacking position of the tray to form a superimposed picture according to the orientation direction of the AR imaging equipment relative to the tray and the stacking position of the article on the tray. The device receives the superimposed picture that the host computer sent and presents to the user is at the pile up neatly article of the pile up neatly position of tray, consequently can solve the manual stacking nonstandard and uncontrollable condition.

In another embodiment of the present invention, the apparatus may further include an information reminding unit (not shown in the figure), wherein the information reminding unit is configured to receive the palletizing upper limit information sent by the upper computer, so as to remind a user of ending the palletizing object. For example, the user may be alerted to the end of palletizing an item by voice or by an image.

Fig. 8 is a schematic structural diagram of an embodiment of the upper computer of the present invention. The upper computer includes a tray image receiving unit 810, an overlay screen generating unit 820, and an overlay screen transmitting unit 830, in which:

the tray image receiving unit 810 is used to receive tray image information acquired by the AR imaging device. The overlay screen generating unit 820 is configured to generate an overlay screen in which the article is located at the stacking position of the tray, based on the stacking position of the article on the tray. The upper computer can determine the stacking position of the article on the tray according to the size information of the article and the size information of the tray, and then generates a superposition picture of the article at the stacking position of the tray by utilizing the AR technology. The overlay screen transmitting unit 830 is configured to transmit the overlay screen in which the object is located at the stacking position of the tray to the AR imaging device, so that the user stacks the object at the stacking position of the tray through the overlay screen presented by the AR imaging device. The virtual images of the articles are seen at the relative positions of the trays by the AR imaging equipment worn by the user, and then the articles can be stacked by the user according to the virtual images of the articles at the relative positions of the trays, so that the articles are stacked orderly.

In this embodiment, the upper computer generates the superposed pictures of the stacked positions of the objects on the tray according to the stacked positions of the objects on the tray, and sends the superposed pictures to the AR imaging equipment so that the objects can be stacked at the stacked positions of the tray through the superposed pictures presented by the AR imaging equipment by a user, and therefore the situation that manual stacking is not standard and uncontrollable can be solved.

Fig. 9 is a schematic structural diagram of another embodiment of the upper computer of the present invention. The upper computer includes a tray image receiving unit 910, a position information receiving unit 920, an attitude evaluation unit 930, a BCR information receiving unit 940, a virtual image generating unit 950, an overlay screen generating unit 960, and an overlay screen transmitting unit 970, wherein:

the tray image receiving unit 910 is configured to receive tray image information acquired by the AR imaging apparatus. The position information receiving unit 920 is configured to receive position information transmitted by the AR imaging apparatus. The posture evaluating unit 930 is configured to perform posture evaluation on the AR imaging device according to the position information of the AR imaging device and the tray image information, and determine the azimuth direction of the AR imaging device relative to the tray. The BCR information receiving unit 940 is used for receiving the item bar code sent by the BCR. The virtual image generating unit 950 is configured to identify an article from an article barcode and generate a virtual image of the article. The overlay screen generating unit 960 is configured to render a virtual image of the item onto the stacking position of the tray to form an overlay screen according to the orientation of the AR imaging device with respect to the tray and the stacking position of the item on the tray. The upper computer further comprises an article volume receiving unit 980 and a stacking position determining unit 990, wherein the BCR information receiving unit 950 is further configured to receive a counting number of the article sent by the BCR. The article volume receiving unit 980 is used for receiving the size information of the article sent by the volume measuring device. The stacking position determining unit 990 serves to determine a stacking position of the article on the tray based on the image information of the tray and the count number and size information of the article. The overlay picture sending unit 970 is configured to send the overlay picture of the stacked position of the tray where the article is located to the AR imaging device, so that the user stacks the article at the stacked position of the tray through the overlay picture presented by the AR imaging device.

In addition, the upper computer may further include an upper limit information sending unit (not shown in the figure), and the upper limit information sending unit is configured to send the palletizing upper limit information to the AR imaging device, so as to remind a user to end palletizing the articles through the AR imaging device. Wherein, can set up the upper limit of putting things in good order of every kind of tray in the host computer, the setting of upper limit is based on two aspects of height and weight. For example, the requirements of the equipment in the current storehouse on pallet goods are that the height is less than or equal to 1.5 meters, and the weight is less than or equal to 300 kilograms, so that the upper computer can simultaneously calculate two conditions when the goods are stacked, and any one condition is met first, namely the upper limit of stacking is reached. Then, the operator is informed of reaching the upper stacking limit through the AR imaging equipment, and stacking is stopped.

In this embodiment, the host computer receives tray image information and positional information that AR imaging device sent to carry out the gesture aassessment to AR imaging device, confirm AR imaging device and for the azimuth of tray directional, the virtual image of article that will generate is played up and is formed the stack on the pile position of tray, and send to AR imaging device so that the stack that the user appears through AR imaging device pile up neatly the article at the pile position of tray position, consequently can solve artifical pile up neatly and the uncontrollable condition.

FIG. 10 is a schematic structural diagram of an embodiment of the artificial assistance system based on AR technology according to the present invention. The system comprises an AR technology based artificial assistant 1010, i.e. an AR imaging apparatus, and an upper computer 1020, wherein the AR technology based artificial assistant 1010 and the upper computer 1020 are described in detail in the above embodiments and are not further explained here. According to the invention, a user can see the virtual images of the commodities at the stacking position of the tray by utilizing the AR technology, and then can stack the objects according to the virtual images, so that the objects can be stacked orderly.

FIG. 11 is a schematic structural diagram of an artificial assistant system based on AR technology according to still another embodiment of the present invention. The system includes a memory 1110 and a processor 1120, wherein:

memory 1110 may be a magnetic disk, flash memory, or any other non-volatile storage medium. The memory is used for storing the instructions in the embodiments corresponding to fig. 1, 2 and 3.

Processor 1120, coupled to memory 1110, may be implemented as one or more integrated circuits, such as a microprocessor or microcontroller. The processor 1120 is configured to execute instructions stored in the memory, and can solve the problem of inaccurate palletizing position by using AR technology.

In one embodiment, as also shown in FIG. 12, the artificial assistance system 1200 based on AR technology includes a memory 1210 and a processor 1220. Processor 1220 is coupled to memory 1210 through a BUS 1230. The artificial assistant system 1200 based on AR technology may be further connected to an external storage device 1250 through a storage interface 1240 for invoking external data, and may be further connected to a network or another computer system (not shown) through a network interface 1260, which will not be described in detail herein.

In the embodiment, the data instructions are stored in the memory and processed by the processor, so that the problem of inaccurate stacking position can be solved by using the AR technology.

In another embodiment, a computer-readable storage medium has stored thereon computer program instructions which, when executed by a processor, implement the steps of the method in the embodiments corresponding to fig. 1, 2, 3. As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable non-transitory storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

Thus far, the present invention has been described in detail. Some details well known in the art have not been described in order to avoid obscuring the concepts of the present invention. It will be fully apparent to those skilled in the art from the foregoing description how to practice the presently disclosed embodiments.

The method and apparatus of the present invention may be implemented in a number of ways. For example, the methods and apparatus of the present invention may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order for the steps of the method is for illustrative purposes only, and the steps of the method of the present invention are not limited to the order specifically described above unless specifically indicated otherwise. Furthermore, in some embodiments, the present invention may also be embodied as a program recorded in a recording medium, the program including machine-readable instructions for implementing a method according to the present invention. Thus, the present invention also covers a recording medium storing a program for executing the method according to the present invention.

Although some specific embodiments of the present invention have been described in detail by way of illustration, it should be understood by those skilled in the art that the above illustration is only for the purpose of illustration and is not intended to limit the scope of the invention. It will be appreciated by those skilled in the art that modifications may be made to the above embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.

Claims (16)

1. An artificial assistance method based on an Augmented Reality (AR) technology is characterized by comprising the following steps:

the AR imaging equipment sends the acquired tray image information to an upper computer;

the upper computer generates a superposed picture of the articles at the stacking positions of the trays according to the stacking positions of the articles on the trays;

and the AR imaging equipment presents the superposed picture of the object positioned at the stacking position of the tray and sent by the upper computer to a user, so that the user can stack the object at the stacking position of the tray according to the superposed picture presented by the AR imaging equipment.

2. The method of claim 1, further comprising:

the AR imaging equipment sends the position information of the AR imaging equipment to the upper computer;

and the upper computer evaluates the posture of the AR imaging equipment according to the position information of the AR imaging equipment and the image information of the tray, and determines the direction and the direction of the AR imaging equipment relative to the tray.

3. The method according to claim 2, wherein the upper computer generates the superimposed picture of the article at the stacking position of the tray according to the stacking position of the article on the tray, including:

the upper computer receives an article bar code sent by bar code scanning equipment BCR, identifies the article according to the article bar code and generates a virtual image of the article; and rendering the virtual image of the article to the stacking position of the tray to form the superposed picture according to the orientation direction of the AR imaging equipment relative to the tray and the stacking position of the article on the tray.

4. The method of claim 3, further comprising:

the upper computer receives the counting number of the article sent by the BCR and the size information of the article sent by the volume measuring device; and determining the stacking position of the articles on the tray according to the image information of the tray and the counting number and the size information of the articles.

5. The method of any of claims 1-4, further comprising:

and the AR imaging equipment receives stacking upper limit information sent by the upper computer so as to remind the user of finishing stacking the objects.

6. An artificial assistance apparatus based on an Augmented Reality (AR) technology, comprising:

the tray image sending unit is used for sending the acquired tray image information to the upper computer so that the upper computer can generate a superposed picture of the articles at the stacking position of the tray according to the stacking position of the articles on the tray;

the stacking drawing receiving unit is used for receiving a stacking drawing of the article at the stacking position of the tray, which is sent by the upper computer;

and the superposed picture presenting unit is used for presenting the superposed picture to a user so that the user can stack the objects at the stacking position of the tray according to the superposed picture presented by the AR imaging equipment.

7. The apparatus according to claim 6, further comprising a location information transmitting unit;

the position information sending unit sends the position information of the AR imaging equipment to the upper computer, so that the upper computer can conveniently evaluate the posture of the AR imaging equipment according to the position information of the AR imaging equipment and the tray image information, determine the direction of the AR imaging equipment relative to the tray, and render the virtual image of the object to the stacking position of the tray to form the stacking picture according to the direction of the AR imaging equipment relative to the tray and the stacking position of the object on the tray.

8. The device according to claim 6 or 7, further comprising an information reminding unit;

the information reminding unit is used for receiving stacking upper limit information sent by the upper computer so as to remind the user of finishing stacking the objects.

9. A host computer, comprising:

the tray image receiving unit is used for receiving tray image information acquired by the AR imaging equipment;

the stacking image generating unit is used for generating a stacking image of the article at the stacking position of the tray according to the stacking position of the article on the tray;

and the superposed picture sending unit is used for sending a superposed picture of the object at the stacking position of the tray to the AR imaging equipment so that a user can stack the object at the stacking position of the tray according to the superposed picture presented by the AR imaging equipment.

10. The upper computer according to claim 9, further comprising a position information receiving unit and a posture evaluating unit;

the position information receiving unit is used for receiving the position information of the AR imaging equipment sent by the AR imaging equipment;

the posture evaluation unit is used for carrying out posture evaluation on the AR imaging device according to the position information of the AR imaging device and the tray image information and determining the orientation direction of the AR imaging device relative to the tray.

11. The upper computer according to claim 10, further comprising a bar code scanner BCR information receiving unit and a virtual image generating unit;

the BCR information receiving unit is used for receiving the article bar code sent by the BCR;

the virtual image generating unit is used for identifying the article according to the article bar code and generating a virtual image of the article;

the superposed picture generating unit is used for rendering the virtual image of the article to the stacking position of the tray to form the superposed picture according to the orientation direction of the AR imaging device relative to the tray and the stacking position of the article on the tray.

12. The upper computer of claim 11, further comprising an item volume receiving unit and a stacking position determining unit;

the BCR information receiving unit is used for receiving the counting number of the article sent by the BCR;

the article volume receiving unit is used for receiving the dimension information of the article sent by the volume measuring device;

the stacking position determining unit is used for determining the stacking position of the article on the tray according to the image information of the tray and the counting number and the size information of the article.

13. The upper computer according to any one of claims 9 to 12, further comprising an upper limit information transmitting unit;

the upper limit information sending unit is used for sending stacking upper limit information to the AR imaging equipment so as to remind the user of finishing stacking the objects through the AR imaging equipment.

14. An artificial assistance system based on Augmented Reality (AR) technology, comprising the artificial assistance device based on AR technology of any one of claims 6 to 8 and the upper computer of any one of claims 9 to 13.

15. An artificial assistance system based on Augmented Reality (AR) technology, comprising:

a memory; and

a processor coupled to the memory, the processor configured to perform the method of any of claims 1-5 based on instructions stored in the memory.

16. A computer-readable storage medium, on which computer program instructions are stored, which, when executed by a processor, implement the steps of the method of any one of claims 1 to 5.