Determining Customs Tariff Numbers from Images: Science Fiction or Soon Reality?

The goal of image recognition is to correctly classify the image at hand into the customs tariff using machine learning.

Machine learning is independent learning by a computer system. Computer systems are given the ability to acquire knowledge on their own through certain specifications by a human being.

In order to use image recognition, a large amount of data is needed. The data must represent the articles to be classified as well as possible. Each image must have the appropriate customs tariff number. This data is called training data. It allows a neural network to be built. This network consists of several layers, all of which have different tasks. Each layer is linked to its predecessor layer and its successor layer. This allows information to be passed on.

There are different types of layers. For example, one layer transforms the image from a two-dimensional to a one-dimensional representation. The last layer contains a list of all possible customs tariff numbers. All the information from the previous layers arrives there, and for each tariff number, the probability that it is the correct number is stored. The customs tariff number with the highest probability is then output.

If you now enter the existing images and the matching customs tariff numbers into the network, the computer starts to “learn”. The individual images are analysed and connections are made between images of products that have the same customs tariff number. Once all the data has passed through the network, you have a trained neural network. Depending on the amount of images available, this can take a few minutes or hours up to several days.

To check how well the network works, you need another set of data, namely the test data. This data is completely alien to the network. It also contains an image of the product and the corresponding customs tariff number. This data also all passes through the network. The quality of the network’s predictions are analysed in the processes. This is done by comparing the predicted customs tariff number with the correct customs tariff number. The more often these two numbers match, the better the network. It is very important that unknown data is used for testing. This is the only way it can become clear whether the network has really learned the connections between the structure of the image and the customs tariff number or has just memorised the training data. This is usually measured by the accuracy of the predictions. When the accuracy is high enough, the network can be used.

Images that have not yet been assigned a customs tariff number can now be placed in this trained network. The correct customs tariff number can now ideally be assigned to the new image through the knowledge acquired by the network from the many example images.

One problem could be the multitude of customs tariff numbers. The network has to decide between an enormous number of customs tariff numbers here. There may be inaccuracies in the prediction due to the fact that the customs tariff numbers are so specific and attention must be paid to the smallest details of the products, which may not even be recognisable in images.
For example, a praying angel is classified as a Christmas item, while a non-praying angel is classified as an ornamental item. Recognising such details and interpreting them correctly is a challenge for the system.

Another hurdle could also be the uneven distribution of images in the BTI database. While there are several thousand images in the database for some customs tariff numbers, there are only a few images for others. This could affect the learning process of the system and therefore the accuracy of image recognition.

However, our image recognition expert Lena Ugele is positive about what the future holds here. She is currently working with her university to implement the project at Grenzlotsen. Lena sees great potential in the new technology and can imagine a hybrid method in the near future, where the system automatically assigns several digits of the customs tariff number to each image and the result is then manually checked for accuracy.

How well the use of image recognition works to determine the customs tariff number remains to be seen in future – we will stay tuned and keep you informed about new developments.