Fraud Detection

Detecting and preventing fraud is one of the biggest issues in online commerce. Fraudulent activity consists of purchasing goods from an online shop without paying for them. The Internet offers scammers the anonymity to remain undetected while they operate. A further complicating factor for online retailers is that customers demand ever shorter periods between ordering and receiving the goods. This inevitably leads to increasingly less time remaining to control online retail orders. To address this issue, we at piazza blu are working on a solution based on machine learning. Based on the historical data of an online shop, the solution could learn which characteristics of an order could indicate fraud. The identified patterns from the historical orders can then be tested on new orders and classified in the course of this test. The foundation for the fraud detection system is a neural network. This network is trained through supervised learning to learn from historical online store orders to be able to classify future orders. Supervised learning is a technique in which a teacher sets a specific outcome for an initial situation. In the case of orders from the online shop, this means that a pre-classification must have been made for each historical order. Thus, it must be known in advance if an order presents a case of fraud or not.

The training method used to teach the neural network is similar to a training session for an inexperienced child. So you can imagine that a child would be shown several orders with the respective results, thus: whether an order is a scam or not. If simple patterns should be found in these orders, in a short time, the child will be able to recognize them and classify unknown orders accordingly. For example, if the orders marked as fraud have a very high total order value, the child could be able to recognize this pattern. If the child is now shown additional orders with a high total order value, it will presumably also, without knowledge of the result, tend to mark this order as fraud. The neural network in the fraud detection system behaves similarly. The order, with all its parameters, is provided to the neural network. For this purpose, the data is prepared such that it can be converted into a numerical value. A neuron is generated for each parameter of the order here. The neurons generated in this manner are the interface of the neural network. During the training, the values of the parameters of an order are assigned to the neurons.

During the training, an output value is then provided for each order. The output values are [0,1] and [1,0], respectively. They stand for “fraud” or “normal shopping”. For each order, the neural network is therefore shown what values the parameters of the order have and to which class the order can be assigned. Using the back-propagation algorithm, the neural network is able to adjust the weights within the network such to improve the approximation of the given classification. This approximation is performed during training for an order in small steps only, since all orders must be taken into account. The training results of the neural network can be graphically displayed using tensorboard, a tool from tensorflow.

In Figure 1, you can see that the deviation of the result which the neural network outputs for an order to the desired result decreases over each iteration of the training. The orange line shows the training set which is used for learning. The graph illustrates that as the number of iterations of training increases, the neural network achieves an increasingly smaller error value and is thus better able to classify the orders into the desired classes. Moreover, no additional improvement is achieved beyond a certain threshold. Training is stopped as soon as further optimization can no longer be achieved after several iterations.

Figure 2 confirms the statement that you could already derive from Figure 1. Here you can see the accuracy with which the neural network can classify the orders from the evaluation set. Again, here you can observe that with the increasing number of iterations in training, an ever greater level of accuracy is achieved. Upon completion of the training, the evaluation set was given a final classification to establish the accuracy with which the neural network can classify the orders. The accuracy achieved on the training data – or the percentage of orders that the neural network was able to classify correctly – was around 96% after training. The neural network can thus classify unknown orders from the online shop with an accuracy of 96%. For 100 orders, 96 can be assigned correctly.

Since the training set consisted of 50% fraud cases and 50% normal purchases, the types of errors made by the neural network were balanced. A confusion matrix was created for the test implementation. For this purpose, another set of orders was classified and for each order it was noted what kind of error the neural network generates when an order has been wrongly classified. There are 2 possibilities here:

Using the confusion matrix, these types of errors can be precisely calculated. For this purpose, the type of error of the neural network per order is recorded. The values of the confusion matrix verify that the types of errors of the neural network are balanced. Thus, the number of fraud cases classified as normal purchases is approximately equal to the number of normal purchases classified as fraud cases.