Follow the latest in AI
There are three ways to control access to s3 bucket and its objects Using bucket policies. Using bucket Access Control Lists (ACL) Using User policies ACL is used only in cases where Objects are not owned by the bucket owner. I.e. objects are uploaded by another account and the bucket owner does not own these … Read more
In this blog we will talk about S3 pricing. Important points to consider are: Pricing is based on region. In this blog we will look at pricing for Sydney region. Pricing varies based on storage used. The more storage you use, the lesser is the price per GB. Here’s the storage cost for 1000G in … Read more
Before we describe what Batch Normalisation is, here are a few introductory terms Internal covariate shift Stochastic Gradient Descent uses a minibatch of input to train the parameters of a layer. The input to a layer is the output from the previous layer. A change in the parameters of the previous layer causes a change … Read more
Why Distributed Machine Learning? In the previous article we looked at how GPGPU, ASICS, AWS’s Inferentia, the new NVidia A100 chip and other advances in hardware have tremendously improved the performance of Machine Learning training and inference. However the increase in the volume of data and the increasing complexity of the machine learning models require … Read more
Machine learning algorithms vary in size from a few parameters to a few billion parameters (e.g. GPT-3). The training data ranges from a few hundred training row to millions of rows. Training a model on a single CPU is not always efficient and so people started using GPU. GPU vs CPU vs GPGPU Wait a … Read more
Amazon Athena is a serverless query engine that allows you to query data in Amazon S3 using standard SQL. You can also use the query result to create another table. This table can then be used as a source of further analytics or reporting. For example : You can use Athena to perform feature engineering … Read more
Amazon SageMaker helps data scientists and Machine Learning developers build, train and deploy machine learning models. It includes Jupyter notebook to build and train model as well SageMaker API to train and deploy model with a few lines of code. Amazon CloudFormation helps in provisioning AWS resources using code. It automates provisioning and configuring resources … Read more
Video Text In this video we will deploy a docker image containing a static website to Kubernetes. This will be a very short demo just to give you a feel of how Kubernetes looks. In a later section we will go through the process of creating the docker image but for now just enjoy the … Read more
If you are setting up a Kubernetes cluster on AWS then you would probably want a cluster that is not accessible to the world. You do that by toggling off public access while creating the cluster, however a problem with that is the DNS Resolution for EKS with private endpoint. The only problem with this … Read more
Kubernetes definition Kubernetes is an open source system for managing containerized applications and provides mechanisms to deploy, maintain and scale them accross multiple hosts spread accross on premise and cloud. It is more than a container orchestration platform because it also takes care of managing the compute, network and storage infrastructure What problems does Kubernetes … Read more