Machine Learning vs Rules Based Systems

There is a famous saying that goes “Learn the rules like a pro, so that you can break them like an artist.”

It’s certainly true in today’s world, where everyone is so fond of breaking rules. Let’s use a simple example: a child eager for cookies. You want to make sure that he doesn’t eat more than four cookies in a day. To ensure that, you’ve set a rule that he can take only four cookies from the box at a time or he’ll get in trouble. The kid tries to take five or six cookies, and you send him to time out; the next time he tries taking three cookies, and you don’t pay much attention. It’s not too much for you, and not against your rule either. Now, the kid starts to understand the rule, your level of attention to different cases, your threshold and the chances of being caught, and he tries to find a way around it.

How do Rules-based Systems Work?

This time, he tries two cookies, then three, then two again until it’s too late for you to realize that the box is already empty. Now, you have to change the rule. This is the fundamental flaw in a rules-based system; rules are predictable, can be circumvented, and require constant manual intervention for improvement.

But now think about how a machine would handle this case, understanding the pattern and behavior of the child, the total cookies in the box, the frequency of attempts, successes, failures, and threshold crosses, and the list goes on. With every cookie-sneaking attempt, the rule changes and the machine behaves differently. Will that confuse the kid? Yes.

Will he still keep trying? Yes—he’s a kid, after all.

Will he succeed in getting past the machine after taking four cookies? NO!

In the same way that the child figures out the rules and sneaks past you for cookies, fraudsters can learn to bypass security-based rules in an effort to defraud banks or other institutions. Security based on self-evolving, self-improving machine learning platforms, however, can prevent fraudsters from circumnavigating your systems.

Why Should Businesses Avoid Rules-Based Systems?

In other words: Why use machine learning?

For years, the world has been following the same logic, “If X, then do Y, else if A, then do B.” Does this work? Yes—that’s the elementary thing we learn in any language. But let’s increase the complexity and put this to the test in the real world. Say you are a money lending company, and to understand the credibility and eligibility of a customer, one has to pass through the 1,000 rules you have defined using “if” conditions.

Now consider the following situations:

• You have to change one of the rules somewhere in the middle of the list. You have to go through a lot of them to find the right one and repeat this process each time a rule needs a change.
• You have to add another 1,000 rules. You start to add to the list, and after a while the list becomes too difficult to manage and has a lot of redundancies.
• If you have one rule out of place, it can cause a number of false positives, which can be a big loss to the business.
• The person who wrote the initial rules for you leaves, and you have to spend time and resources to catch up on the long list of rules.
•  The rules are difficult to trace back in the event of any exceptions, and a person needs to know all the rules that are in place in order to do this.

Do you see the problems here? When every company wants to build scalable systems to handle terabytes of data, it’s not feasible to stick with rules-based decision systems.

Traditional Programming vs Machine Learning

As the world of data expands, it’s time to look beyond binary outcomes by using a probabilistic approach rather than a deterministic one. For example, what happens if you ask your system a question about a customer’s loan repayment?

A deterministic system will put in all the factors as per the rules and tell you whether the person will repay the loan. A probabilistic system can give you wider insights into how much of the loan the person is likely to repay, or the probability of the person committing fraud and hanging you out to dry. The probabilistic system relies on the world of machine learning.

Instead of “based on the inputs you have given me, here is the result,” machine learning works on the concept of “based on historic outcomes in this situation, here is what we can say about the future.”

Machine learning takes a probabilistic approach by using historical data and outcomes. It considers not only the input but n number of other factors. Machine learning understands patterns and trends in historic data, and gives you the probability of different outcomes based on that.

Machines Need to Be Trained

The big question is, How does a machine become informed about historic events? That process is called training.

How Machine learning works: models that rely on algorithms that can be set totally data-driven. For these algorithms to work, the models have to be trained on historic data about outcomes that have already been identified or labeled by human experts.

In the long run, machine learning is better than rules-based systems because of the models that can adapt to changing trends and the flexibility to tweak the parameters involved. The less human involvement there is in defining rules ensures that the system is less predictable and hence harder to penetrate.

When it comes to optimizing outcome and accuracy of any system, consider ditching those unwieldy rules-based systems and implementing a machine learning platform.