Everything You Need to Know About Compositional Learning in Machine Learning

Machine learning is a critical aspect of modern businesses and research. Using algorithms and neural networks, machine learning assists the computer system in improving their performance. They build models using simple data that helps in making decisions without needing to be programmed.

The first case of neural networks took place in 1943, when a mathematician Walter Pitts and a neurophysiologist Warren McCulloch wrote a paper on neurons and how they function. This was when they decided they would create a model of this with the help of an electrical circuit, thus the birth of the neural network.

Further research on the Turing test was done where computers had to convince humans that it is a human and not a computer. Decades later, Google’s DeepMind AI game AlphaGo could successfully defeat the world’s number one player in the Go games, an ancient board game that is complex and is said to have a more possible configuration for pieces than the atoms present in the universe.

Undoubtedly, AI and machine learning have progressed to greater heights one cannot fathom, this ideally is driven by computational power.

As the thrive gets greater in creating computer chips in a classical computing manner, you will find more bits that are nearing the smallest molecular size. Therefore, machine learning cannot rely on computational power anymore to create powerful models. The reason why machine learning now turns to compositional learning.

Defining compositional learning?

The idea behind compositional learning states that one single model cannot do all the tasks. For example, when deep neural networks were used for one single task like classifying an image into a cat or a dog or recognizing cancer, they performed extremely well. Sadly, it has been observed that the model could perform only one task at a time.

As technology in AI and applications become more complex, the single neural network will only get larger. Once this happens, it may further account for more complications with more neurons. Thus, the ability to grow might reach a dead end.

Now if you combine all these neural networks to perform certain segments of a complete task, as a whole the model tends to perform better even while taking up intricate tasks, maintaining a reasonable computing space.

If these tasks are broken down into fragments (several neural networks), each of these networks can specialize in a certain field. This relationship is almost similar to asking the prime minister to make decisions with or without any support from the secretaries of defense, health, labor, or any other departments.

Here’s a simple example,

ai chatbot conversation with human
Source: andre-ye.medium

A chatbot is developed to help upscale a restaurant’s business by engaging with customers or performing tasks like making a reservation or inquiring about the menu, or having general chitchat. The conversation can be easily broken into three sections – chitchat, information retrieval, and action are taken.

Well, instead of having one single model that takes in the customers’ reactions and sends an output – response, we can easily opt for a more distributed system.

Single model: current message history > general neural network (encoder – decoder) > response

Compositional model: current message history > task distributor neural network >

  1. Pleasantries network > response
  2. Info retrieval network > response
  3. Action network > response

Source: andre-ye.medium

With the help of a distributed model, you get two benefits:

  1. Faster runtime: though training distributed models can be a challenging process, they work even much faster while making predictions that require quick responses. This is possible because the distributed model can be thought of as splitting a singular model – thus, making the information pass through useful neurons that contain the current task instead of needing it to flow throughout the entire network.
  2. Higher accuracy: the tasks have been delegated to three separate models, each of which specializes in their field improving the performance of the model.

Source: andre-ye.medium

Encoder-decoder networks or GANs (generative models) are composed of multiple networks and at times can be said as compositional models. However, in the current context, they will still be considered as a singular model since compositional models expand making it more effective. The compositional model we’re talking about here is described as the ‘composition-composition model’.

One of the major reasons why compositional learning works perfectly well is perhaps because our brains are also compositional in nature. To be precise, compositional learning is much more difficult than standard modeling.

However, we’re yet to unveil what AI has in store for compositional learning.

Start your journey of knowledge with brainstorming box. Our mission is to make learning easier and Interesting than it has ever been. Each day, we curate fascinating topics for those who pursue knowledge with passion.

What is Data Labeling and What is the Role of a Data Labeler ?

A driverless car should be faultless – there is no room for error. The ability of a driverless car’s accuracy is improved only if the...

NASA Discovered Earth-Like Alien Planets Orbiting a Sun-Like Star

In an era where we seek signs of life beyond the earth’s confinement is a serious issue. However, scientifically it is interesting. It isn’t...

7 Types of Artificial Intelligence

when you think about artificial intelligence you might have imagined self- driving cars or robots overtaking humans as shows in Sci-Fi...

Top 10 Real World Applications of Artificial Intelligence in 2020

Artificial intelligence! It is amazing how this fine technology is taking the world by storm. From augmenting skills for the future workforce to providing...

Quantum Supremacy is Underway: Why You Should Care?

Quantum computing! Is it still an enigma to the world? Ever heard of advanced math machines? These can also be called the quantum computer. Quantum...

Neuro-Symbolic AI: Machines to Build Reasoning Power Like Humans

Perhaps you’ve heard about neural networks, software simulations interconnected inside a computer that helps learn things, recognize patterns, and make decisions in the same...