Distilling knowledge from a Teacher to a Student in a Deep Neural Network
In this article, you will learn.
- An easy to understand explanation of Teacher-Student knowledge distillation neural networks
- Benefits of Knowledge distillation
- Implementation of Knowledge distillation on the CIFAR-10 dataset
Overview of Knowledge Distillation
Large deep neural networks or an ensemble of deep neural networks built for better accuracy are computationally expensive, resulting in longer inference times which may not be ideal for near-real-time inference required at the Edge.
The training requirements for a model are different from requirements at the inference. …
A simplistic explanation of what is Blockchain and how it works
You are in a grocery store and scan the QR code on your favorite coffee. You can view the details of where the coffee beans grew, cleaned, triaged, processed, and then roasted. what-if you can trace the supply and distribution to know the means of transport and how long it took to arrive at the store and so forth
Is there a technology that can help record information at each step and make that information available to everyone transparently?
Welcome to the world of Blockchain
Blockchain is an immutable, distributed, decentralized; peer-to-peer ledger replicated across multiple nodes connected in a…
A Stochastic Optimization Technique Inspired by Nature
Particle Swarm Optimization(PSO) is a meta-heuristic stochastic nonlinear optimization algorithm proposed by Kennedy and Eberhart in 1995 to solve computationally challenging optimization problems.
Inspiration
Particle Swarm Optimization(PSO) is inspired by nature. Flocks of bird and fish schools adjust their physical movement to avoid predators, seek food or mates, or optimize environmental parameters such as temperature.
Birds and fishes use swarm intelligence, social behavior, and movement dynamics to share information among their group for a greater survival advantage.
Objective
The PSO optimization objective is to find the optimum function that either maximizes the accuracy or precision or minimizes the loss.
The objective of PSO is to find a global optimum over a search space. A swarm in the algorithm is represented by a set of particles…
Explore and Understand Visual Transformers to perform Image Recognition
You will explore
- Strength and weaknesses of the current Convolutional Neural Network
- How Visual Transformers resolve the weakness in CNN by understanding its working
- Different variations of Visual Transformers.
Strength and Weaknesses with Convolutional Neural Network(CNN)
Strength
- CNN has inductive biases like translation invariance and locally restricted receptive field. Translational invariance recognizes an object even when its appearance varies, like changing orientation or zoom in or zoom out.
Weakness
- Specifically designed for images.
- It has a domain-specific design and is not scalable for it to be domain agnostic. CNN uses pixel arrays where each pixel represents varying importance and is domain-specific.
- CNN lacks a global understanding of…
Implement Facial Matching Algorithm using Siamese Network in TensorFlow
My previous post was an Intuitive explanation of the Siamese Network, and in this post, it is the implementation of the Siamese network for Facial Recognition in TensorFlow.
Facial Recognition
Facial Recognition is to identify or verify people in the images or videos. It is the task to match faces to a given face or finding similar faces from a given facial dataset.
Siamese Network is a One-shot classifier with two mirror image subnetworks to rank similarity or dissimilarity between the two inputs using Similarity score.
The Density-based Unsupervised Clustering algorithm robust to outliers
You will learn what DBSCAN is, how it works, the pros and cons of DBSCAN, and finally, implementation.
What is DBSCAN?
DBSCAN is Density-Based Spatial Clustering for Applications with Noise, an unsupervised clustering algorithm which finds core sample of high density and expands cluster from them
- Clusters data into a high-density area separated by a low-density area.
- Clusters formed by DBSCAN can be any shape and not necessarily convex-shaped
- It is an unsupervised, deterministic clustering algorithm and gives the same clusters back with the same data in the same order; however, when the order of the data is changed, you may get a…
An intuitive explanation of Siamese Network
You have a new account holder in your bank and would like to set his signature for verification. You have only one sample signature from the member. How can you use a neural network to perform the signature verification?
To find the answer, read on…
Siamese network inspired by Siamese twins has a unique architecture to naturally rank similarity or dissimilarity between inputs.
Key features of Siamese Network
- Siamese network takes two different inputs passed through two similar subnetworks with the same architecture, parameters, and weights.
- The two subnetworks are a mirror image of each other, just like the Siamese twins. Hence, any change to…
Perform fraud detection using Autoencoders in TensorFlow
Learn what are AutoEncoders, how they work, their usage, and finally implement Autoencoders for anomaly detection.
AutoEncoder is a generative unsupervised deep learning algorithm used for reconstructing high-dimensional input data using a neural network with a narrow bottleneck layer in the middle which contains the latent representation of the input data.
Autoencoder consists of an Encoder and a Decoder.
- Encoder network: Accepts high-dimensional input data and translates it to latent low-dimensional data. The input size to an Encoder network is larger than its output size.
- Decoder network: The Decoder network receives the input from the Encoder coder’s output. Decoder’s objective…
Explore One-Class SVM for Anomaly detection
This post is the second in the series; here you will explore One-Class SVM, a semi-supervised technique for anomaly detection.
Anomaly detection using Machine Learning can be divided into Supervised, Semi-Supervised, or Unsupervised algorithms.
- Supervised Anomaly Detection: A labeled dataset with inliers and outlier data points where learning happens based on the labeled dataset used for training.
- Semi-Supervised Anamoly Detection(Novelty Detection): Outliers do not pollute training data, and the anomaly detection algorithm only detects if the new observation is an inlier or an outlier. …
Explore Interquartile Range, Elliptic Envelope, and Isolated Forest for Anomaly or Outlier Detection
In this post, you will explore supervised, semi-supervised, and unsupervised techniques for Anomaly detection like Interquartile range, Isolated forest, and Elliptic envelope for identifying anomalies in data.
You might have played “Find the odd one out” as a kid where you were given a few data points like Rose, Lily, Tulip, Daffodils, and Egg Plant, and then based on the data pattern, figured out that Egg Plant was the odd one out as all the remaining items are the name of flowers and Egg Plant is a vegetable.
Anomaly detection is very similar to finding the odd one out where…
Renu Khandelwal
Loves learning, sharing, and discovering myself. Passionate about Machine Learning and Deep Learning
