Everything You Need to Know About Compositional Learning in Machine Learning

Machine learning is a critical aspect of modern businesses and research. Using algorithms and neural networks, machine learning assists the computer system in improving their performance. They build models using simple data that helps in making decisions without needing to be programmed.

The first case of neural networks took place in 1943, when a mathematician Walter Pitts and a neurophysiologist Warren McCulloch wrote a paper on neurons and how they function. This was when they decided they would create a model of this with the help of an electrical circuit, thus the birth of the neural network.

Further research on the Turing test was done where computers had to convince humans that it is a human and not a computer. Decades later, Google’s DeepMind AI game AlphaGo could successfully defeat the world’s number one player in the Go games, an ancient board game that is complex and is said to have a more possible configuration for pieces than the atoms present in the universe.

Undoubtedly, AI and machine learning have progressed to greater heights one cannot fathom, this ideally is driven by computational power.

As the thrive gets greater in creating computer chips in a classical computing manner, you will find more bits that are nearing the smallest molecular size. Therefore, machine learning cannot rely on computational power anymore to create powerful models. The reason why machine learning now turns to compositional learning.

Defining compositional learning?

The idea behind compositional learning states that one single model cannot do all the tasks. For example, when deep neural networks were used for one single task like classifying an image into a cat or a dog or recognizing cancer, they performed extremely well. Sadly, it has been observed that the model could perform only one task at a time.

As technology in AI and applications become more complex, the single neural network will only get larger. Once this happens, it may further account for more complications with more neurons. Thus, the ability to grow might reach a dead end.

Now if you combine all these neural networks to perform certain segments of a complete task, as a whole the model tends to perform better even while taking up intricate tasks, maintaining a reasonable computing space.

If these tasks are broken down into fragments (several neural networks), each of these networks can specialize in a certain field. This relationship is almost similar to asking the prime minister to make decisions with or without any support from the secretaries of defense, health, labor, or any other departments.

Here’s a simple example,

ai chatbot conversation with human
Source: andre-ye.medium

A chatbot is developed to help upscale a restaurant’s business by engaging with customers or performing tasks like making a reservation or inquiring about the menu, or having general chitchat. The conversation can be easily broken into three sections – chitchat, information retrieval, and action are taken.

Well, instead of having one single model that takes in the customers’ reactions and sends an output – response, we can easily opt for a more distributed system.

Single model: current message history > general neural network (encoder – decoder) > response

Compositional model: current message history > task distributor neural network >

  1. Pleasantries network > response
  2. Info retrieval network > response
  3. Action network > response

Source: andre-ye.medium

With the help of a distributed model, you get two benefits:

  1. Faster runtime: though training distributed models can be a challenging process, they work even much faster while making predictions that require quick responses. This is possible because the distributed model can be thought of as splitting a singular model – thus, making the information pass through useful neurons that contain the current task instead of needing it to flow throughout the entire network.
  2. Higher accuracy: the tasks have been delegated to three separate models, each of which specializes in their field improving the performance of the model.

Source: andre-ye.medium

Encoder-decoder networks or GANs (generative models) are composed of multiple networks and at times can be said as compositional models. However, in the current context, they will still be considered as a singular model since compositional models expand making it more effective. The compositional model we’re talking about here is described as the ‘composition-composition model’.

One of the major reasons why compositional learning works perfectly well is perhaps because our brains are also compositional in nature. To be precise, compositional learning is much more difficult than standard modeling.

However, we’re yet to unveil what AI has in store for compositional learning.

Start your journey of knowledge with brainstorming box. Our mission is to make learning easier and Interesting than it has ever been. Each day, we curate fascinating topics for those who pursue knowledge with passion.

Rust Found on Moon – and Earth Could Be The Reason

It is indeed bizarre to say the moon is rusting because of the earth’s atmosphere. But wait, there’s no oxygen on the moon and neither...

Why do we weigh different on other planets?

Have you ever thought why do we weight different on other planets . What makes us feel more or less weight...

You’ve Heard of Black Holes. But ‘White Holes’ Might Blow Your Mind

Most of us are aware of the black hole, what does it mean and what exactly it does and most recently...

Discover All You Need to Know About Deep Learning

Deep learning amazed the audience when Google’s AI beat a human top player at the game of Go, an ancient contest, and a game...

What Are Wormholes? Could We Really Travel Through Them?

The human curiosity is one such element of the psyche that simply can never be quenched, especially when it comes to space travel. Did...

What is Data Labeling and What is the Role of a Data Labeler ?

A driverless car should be faultless – there is no room for error. The ability of a driverless car’s accuracy is improved only if the...