AI algorithms and architectures

Trend 1: Improve generalization and accuracy with deep neural network architectures

Adoption of deep learning-based solutions to solve enterprise-class problems is driven by some key factors, such as availability of graphics processing unit computing (GPU), availability of large labeled data, and fast-paced innovations in new deep learning algorithms. They promise higher accuracy and better generalization characteristics as compared to classical algorithms such as SVM, Naive Bayes and Random Forest. However, the need for a large set of labeled data and the cost of GPU computing are still two challenges to the mainstream adoption of deep learning. Transfer learning-based models, have made huge headway in overcoming the limitations of lack of sufficient availability of labeled data and GPU. Addressing certain complex problems in computer vision, NLP and speech domains has become feasible with evolving architecture such as Transformers.

Infosys partnered with a large technology company to transform its existing system that was moderating user uploaded content based on certain pre-configured historical rules and policies to an AI-based moderation. The AI model was trained with a very limited set of label data for supervised transfer learning-based deep neural net architecture for vision and text to identify, classify and isolate any toxic content arriving from user uploaded forms.

As part of a prestigious global tennis tournament, Infosys trained a transfer learning-based computer vision model using a limited set of brand logo images, to establish the amount of time and time frames a particular brand was visible. Similarly, for compiling key moments, player action identification such as waving hands to the crowd was done with a similar approach.

There are several examples illustrated across the following sections where Infosys deployed transfer learning-based techniques for speech, vision and text to overcome labeled data deficit challenges during supervised model training. In some cases, Infosys also used conventional AI algorithms such as logistic regression, SVM to drive early results and then used those results to train a deep learning-based AI model for improved generalization and accuracy.

AI algorithms and architectures

Trend 2: Transition from System 1 deep learning to System 2 deep learning

The current state of deep learning-based AI is referred as System 1 deep learning, and it can be best illustrated with an example of a person driving a car in a known vicinity while talking on the phone or with a passenger, and is able to automatically drive through, without consciously focusing on driving. However, the same person driving through an unknown vicinity will need a lot more focus and will need to use various logical reasoning and connections to reach the destination. These types of problems, which need a combination of reasoning and a sense of on-the-fly decision-making, still can’t be solved with current AI discipline maturity and are considered System 2 deep learning.

System 1 deep learning’s current state is due to certain current limitations of deep learning’s generalization capabilities, where these algorithms

  • are not able to correctly work on (detect) unseen data patterns;
  • need to have balanced distribution of data in training and testing sets;
  • lack the ability to do continuous learning based on changes in enviroments in real time, similar to active agents;
  • lack the logical and reasoning capability to combine high-level semantic concepts, and
  • are unable to deal with out-of-distribution (noise) data.

These are some of the reasons for the current state of AI’s inability to deal with the System 2 deep learning state.

System 2 deep learning is where some of these challenges are being worked upon by leveraging techniques like attention-based architectures and models, multitask learning, incorporating principles of consciousness, and meta learning with an emphasis on unsupervised, zero-shot learning techniques.

Subscribe

To keep yourself updated on the latest technology and industry trends subscribe to the Infosys Knowledge Institute’s publications

Infosys TechCompass