AWS AI Practitioner Certification Cheat Sheet

AWS AI Practitioner Certification exam study notes, exam pointers, and cheat sheet.

Photo by Google DeepMind on Unsplash

This material was gathered during my preparation for the AWS AI Practitioner Certification Exam (here’s my badge). I created and curated this cheatsheet with useful information that will be handy to review before taking the exam.

Note that to pass this exam is crucial to get a good overall understanding of the basic AI concepts and AWS AI-focused services. Use these notes as complementary material and not complete study material for the exam.

AWS frequently changes information, configuration, and options of different services, so some of the content might become outdated at some point. Make sure to cross-check and validate the information you are getting from online sources with the official AWS Documentation and FAQs before your exam.

OK, enough with the disclaimers, let’s get to it.

SageMaker

• Automatic model tuning can be performed using SageMaker Automatic Model Tuning (AMT). It can automatically choose hyper parameter ranges, search strategy, maximum runtime of a tuning job, early stopping type for training jobs, number of times to retry a training job, and model convergence flag to stop a tuning job, based on the objective metric you provide.

SageMaker Inference

• Asynchronous — process smaller payloads without requiring real-time responses by queuing the requests and handling them in the background. This method is cost-effective and efficient when some delay is acceptable, as it frees up resources and optimizes compute usage. Asynchronous inference is ideal for scenarios where the payload size is less than 1 GB and immediate results are not critical.

• Batch — generally used for processing large datasets all at once. it is typically more efficient for handling larger payloads (several gigabytes or more). For smaller payloads of less than 1 GB, batch inference might be overkill and less cost-efficient compared to asynchronous inference.

• Real-time — optimized for scenarios where low latency is essential, and responses are needed immediately

• Serverless — good choice for workloads with unpredictable traffic or sporadic requests, as it scales automatically based on demand

SageMaker Clarify

• Focused on detecting bias in data and explaining model predictions to ensure transparency and fairness.

SageMaker GroundTruth

• Labeling data with a human-in-the-loop

• GroundTruth Plus provides a fully managed data labeling service that helps deliver high-quality annotations. Combination of human labelers and machine learning-assisted labeling to ensure accuracy and consistency in the labels.

SageMaker Canvas

• No-code interface to create ML models

SageMaker Data Wrangler

• Reduces the time it takes to aggregate and prepare tabular and image data for ML from weeks to minutes from a single visual interface.

• Data Wrangler supports several balancing operators as part of the Balance data transform to fix bias by balancing the dataset.

SageMaker Feature Store

• Fully managed, purpose-built repository to store, share, and manage features for machine learning (ML) models. Features are inputs to ML models used during training and inference.

SageMaker JumpStart

• Amazon SageMaker JumpStart is a machine learning (ML) hub that can help you accelerate your ML journey. With SageMaker JumpStart, you can evaluate, compare, and select Foundation Models (FMs) quickly based on pre-defined quality and responsibility metrics to perform tasks like article summarization and image generation. Pretrained models are fully customizable for your use case with your data, and you can easily deploy them into production with the user interface or SDK.

SageMaker Model Dashboard

• Aggregates and displays data from Amazon SageMaker Model Cards, SageMaker Model Monitor and SageMaker Endpoint services. It’s a centralized repository of all models created in your account. The models are generally the outputs of SageMaker training jobs, but you can also import models trained elsewhere and host them on SageMaker.

SageMaker Model Cards

• Document critical details about your machine learning (ML) models in a single place for streamlined governance and reporting. Model cards can help you to capture key information about your models throughout their lifecycle and implement responsible AI practices.

• Catalog details such as the intended use and risk rating of a model, training details and metrics, evaluation results and observations, and additional call-outs such as considerations, recommendations, and custom information

• Describes how a model should be used in production, the scenarios in which is appropriate to use a model, and additional considerations such as the type of data to use with the model or any assumptions made during development.

SageMaker Model Monitor

• Designed to track the performance of machine learning models in production by monitoring data drift, bias, and other deviations. primarily focused on post-deployment model monitoring

Governance in SageMaker

• SageMaker provides purpose-built ML governance tools across the ML lifecycle. With Amazon SageMaker Role Manager, administrators can define minimum permissions in minutes. Amazon SageMaker Model Cards make it easier to capture, retrieve, and share essential model information from conception to deployment, and Amazon SageMaker Model Dashboard keeps you informed on production model behavior, all in one place.

Amazon Augmented AI

• Get human review of low-confidence predictions or random prediction samples. Brings human review of ML predictions to all developers by removing the heavy lifting associated with building human review systems or managing large numbers of human reviewers.

Bedrock

• For testing and deploy customized models for Amazon Bedrock (via fine-tuning or continued pre-training), it is mandatory to use Provisioned Throughput.

Knowledge bases

• Managed opensearch serverless vector store

• Retrieval-Augmented Generation (RAG) use cases. When a customer query comes in, the RAG framework first retrieves the most relevant pieces of information from this knowledge base and then uses an LLM to generate a coherent response based on the retrieved context.

Agents

• Agents are software components or entities designed to autonomously or semi-autonomously perform specific actions or tasks based on predefined rules or algorithms.

Guardrails

• Detects sensitive information such as personally identifiable information (PIIs) in input prompts or model responses. You can also configure sensitive information specific to your use case or organization by defining it with regular expressions (regex).

Amazon Rekognition

• image and video analysis service that makes it easy to add advanced computer vision capabilities to your applications.

• Deep learning, image recognition service, detect objects within an image.

• Offers pre-trained and customizable computer vision (CV) capabilities to extract information and insights from your images and videos

• detect text within images. Rekognition specializes in identifying text located spatially within an image, for instance, words displayed on street signs, t-shirts, or license plates.

Amazon Comprehend

• Get insights about the content of docs

• Detect key phrases, language, sentiment, entities in text, intelligent document processing use cases.

• Search social feeds for mentions

Amazon Textract

• (ML) service that automatically extracts text, handwriting, and data from scanned documents

• It goes beyond simple optical character recognition (OCR) to identify, understand, and extract data from forms and tables.

Amazon Transcribe

• Convert audio input (speech) to text. Using Automatic Speech Recognition (ASR) technology, customers can choose to use Amazon Transcribe for a variety of business applications, including transcription of voice-based customer service calls, generation of subtitles on audio/video content, and conduct (text-based) content analysis on audio/video content.

• Amazon Transcribe Medical is an automatic speech recognition (ASR) service that makes it easy for you to add medical speech-to-text capabilities to your voice-enabled applications.

Amazon Translate

• Text translation service that uses advanced machine learning technologies to provide high-quality translation on demand

Amazon Polly

• Uses deep learning technologies to synthesize natural-sounding human speech, so you can convert articles to speech.

Amazon Personalize

• Fully managed machine learning (ML) service that uses your data to generate product and content recommendations for your users. You provide data about your end-users (e.g., age, location, device type), items in your catalog (e.g., genre, price), and interactions between users and items (e.g., clicks, purchases). Personalize uses this data to train custom, private models that generate recommendations that can be surfaced via an API.

Amazon Forecast

• Fully managed service that uses statistical and machine learning algorithms to deliver highly accurate time-series forecasts.

Amazon Kendra

• A highly accurate and easy-to-use enterprise search service that’s powered by machine learning (ML).

AWS Artifact

• Provide access to a wide range of AWS compliance reports. AWS Artifact allows users to configure settings to receive notifications when new compliance documents or reports are available.

Amazon Lex — AI Chat Builder

• Allows users to interact with any application using natural language voice or chat. Use Amazon Lex to build and deploy conversational AI interfaces for any application. Offers pre-built bots to offer personalized experiences for mobile subscriber activities.

AWS DeepRacer

• The AWS DeepRacer vehicle is a Wi-Fi-enabled, physical vehicle that can drive itself on a physical track by using a reinforcement learning model. Uses Reinforcement Learning

Trainium Instance types

• Designed with energy efficiency in mind, providing optimal performance per watt for machine learning workloads. They offer the best performance at the lowest power consumption, reducing the carbon footprint of training large-scale models.

Security, Compliance, and Governance for AI Solutions

• Each AZ one or more data centers

• Each AWS Region consists of minimum 3 AZs

• For controlling risks instruct the model to stick to the prompt by adding explicit instructions to ignore any unrelated or potentially malicious content

Metrics, Scores, Parameters, Inputs, Tools

Model Pameters

Values that define a model and its behavior in interpreting input and generating responses. Model parameters are controlled and updated by providers. You can also update model parameters to create a new model through the process of model customization. In other words, Model parameters are the internal variables of the model that are learned and adjusted during the training process. These parameters directly influence the output of the model for a given input. Examples include the weights and biases in a neural network.

Hyperparameters

Values that can be adjusted for model customization to control the training process and, consequently, the output custom model. They are external configurations set before the training process begins. They control the training process and the structure of the model but are not adjusted by the training algorithm itself. Examples include the learning rate, the number of layers in a neural network, etc.

Temperature

Temperature is a value between 0 and 1, and it regulates the creativity of the model’s responses. Higher Temperature to get more creative responses**.** Use a lower temperature if you want more deterministic responses.

Top-P

Represents the percentage of most likely candidates that the model considers for the next token. Choose a lower value to decrease the size of the pool and limit the options to more likely outputs. Choose a higher value to increase the size of the pool and allow the model to consider less likely outputs.

Top-K

Top K represents the number of most likely candidates that the model considers for the next token. Choose a lower value to decrease the size of the pool and limit the options to more likely outputs. Choose a higher value to increase the size of the pool and allow the model to consider less likely outputs.

BLEU (Bilingual Evaluation Understudy) score, a metric specifically designed to evaluate the quality of text that has been machine-translated by comparing it with one or more reference translations. Most widely used metrics for evaluating machine translation quality.

ROUGE (Recall-Oriented Understudy for Gisting Evaluation is a metric used mainly for evaluating the quality of automatic text summarization

Accuracy is a broad metric typically used to evaluate classification tasks where the model’s output is compared against the correct label.

Precision, Recall, and F1-Score are standard performance metrics used to evaluate the effectiveness of a classification system
Precision: Measures the accuracy of the positive predictions, calculated as the ratio of true positives to the sum of true positives and false positives.
Recall (Sensitivity): Measures the ability of the classifier to identify all positive instances, calculated as the ratio of true positives to the sum of true positives and false negatives.
F1-Score: The harmonic mean of Precision and Recall, providing a single metric that balances both concerns. When dealing with imbalanced datasets in binary classification

BERT score is a more advanced metric that uses contextual embeddings to assess the semantic similarity between translated and reference texts. It’s designed to capture the contextual meaning of words by looking at both the words that come before and after them

Shapley values are a local interpretability method that explains individual predictions by assigning each feature a contribution score based on its marginal effect on the prediction. This method is useful for understanding the impact of each feature on a specific instance’s prediction.

Partial Dependence Plots (PDP), on the other hand, provide a global view of the model’s behavior by illustrating how the predicted outcome changes as a single feature is varied across its range, holding all other features constant. PDPs help understand the overall relationship between a feature and the model output across the entire dataset.

Confusion matrix is a tool specifically designed to evaluate the performance of classification models by displaying the number of true positives, true negatives, false positives, and false negatives. This matrix provides a detailed breakdown of the model’s performance across all classes, making it the most suitable choice for evaluating a classification model’s accuracy and identifying potential areas for improvement.

Bias is an error introduced by approximating a real-world problem (which may be complex) by a simpler model. High bias can cause the model to miss relevant relations between features and target outputs (underfitting)

Variance is an error introduced by the model’s sensitivity to small fluctuations in the training data. High variance can cause the model to mimic the random noise in the training data rather than the intended outputs (overfitting).

Fundamentals of AI and ML

Generative models learn the underlying patterns of data to create new, similar data, while discriminative models learn to distinguish between different classes of data. Generative models, such as GPT-3, can generate new content, whereas discriminative models are used for classification tasks.

Maintaining data lineage involves tracking the flow, transformations, and origins of data throughout its lifecycle. This is crucial for ensuring data privacy, security, and compliance with regulatory standards.

When the number of samples in the majority class (bigger) is considerably larger than the number of samples in the minority (smaller) class, the dataset is considered imbalanced.

Labeled data is data that comes with predefined labels or annotations. E.g. images labeled with the objects they contain. This type of data is crucial for supervised learning, where the model learns to predict the output from the input data.

In supervised learning, you define both the input and the expected associated output. Supervised learning algorithms learn patterns and relationships between the input and output pairs. Then, they predict outcomes based on new input data. Examples: Linear regression, logistic regression, decision tree, neural network, classification

Unlabeled data is data that does not come with any labels, annotations, or explicit instructions about what it represents. This type of data is often used in unsupervised learning, where the model attempts to find patterns or structures in the data without predefined labels. Examples: Clustering: Grouping similar data points

Unsupervised learning algorithms receive inputs with no specified outputs during the training process. They find hidden patterns and relationships within the data using statistical means. For instance, you could provide a set of documents, and the algorithm may group them into categories it identifies based on the words in the text. You do not get any specific outcomes; they fall within a range. Examples: clustering (K-Means), association rule learning, dimensionality reduction, probability density

Semi-supervised learning is when you apply both supervised and unsupervised learning techniques to a common problem. This technique relies on using a small amount of labeled data and a large amount of unlabeled data to train systems. First, the labeled data is used to partially train the machine learning algorithm. After that, the partially trained algorithm labels the unlabeled data. This process is called pseudo-labeling. The model is then re-trained on the resulting data mix without being explicitly programmed. Examples, fraud detection, sentiment analysis, document classification.

Reinforcement learning mimics the trial-and-error learning process that humans use to achieve their goals. Software actions that work towards your goal are reinforced, while actions that detract from the goal are ignored. Excels in complex environments and requires less human interaction. RL has a predetermined end goal. While it takes an exploratory approach, the explorations are continuously validated and improved to increase the probability of reaching the end goal. It can teach itself to reach very specific outcomes.

Overfitting occurs when the model is overly complex and captures noise or random fluctuations in the training data rather than the underlying patterns.

By fine-tuning hyperparameters, such as increasing regularization or early stopping or adjusting dropout rates, the model can avoid overfitting to the training data and better generalize to new, unseen data in production. This approach helps improve the model’s performance across various data distributions.

To prevent over-fitting, techniques such as cross-validation, regularization, and pruning are employed. Cross-validation helps ensure the model generalizes well to unseen data by dividing the data into multiple training and validation sets. Regularization techniques, such as L1 and L2 regularization, penalize complex models to reduce overfitting. Pruning simplifies decision trees by removing branches that have little importance.

Increasing the number of epochs allows the model to learn from the training data for a longer period, potentially capturing more complex patterns and relationships, which can improve accuracy. Multiple epochs are run until the accuracy of the model reaches an acceptable level, or when the error rate drops below an acceptable level.

Transfer learning allows a model to utilize the knowledge learned from one task or dataset to improve its performanceon a new, but related task.

Incremental training is useful for updating a model with new data continuously. it focuses on enhancing a single model’s performance with its own data. It’s designed to help models adapt to new data over time without forgetting previously learned knowledge

Deep Learning, model training involves feeding large datasets into the neural network and adjusting the weights and biases through multiple iterations.

Computer vision focuses on interpreting and understanding the content of images to make decisions, such as object detection, facial recognition, and scene understanding

Generative Adversarial Networks (GANs) work by training two neural networks in a competitive manner. The first network, known as the generator, generates fake data samples by adding random noise. The second network, called the discriminator, tries to distinguish between real data and the fake data produced by the generator. They are specifically designed for generating synthetic data that is statistically similar to real data

Convolutional Neural Network (CNN) are designed for tasks such as image and video recognition, object detection, and similar applications involving grid-like data (such as pixels in an image)

Recurrent Neural Networks (RNNs) are typically used for sequence data, such as time series or natural language processing tasks (video analysis)

WaveNet is tailored for audio data generation, specifically for tasks such as speech synthesis and audio signal processing.

Support Vector Machines (SVMs) are used for classification and regression

Exploratory Data Analysis (EDA) phase, which involves examining the data through statistical summaries and visualizations to identify patterns, detect anomalies, and form hypotheses.

Foundation Models provide a broad base with generalized capabilities that can be applied to various tasks such as natural language processing (NLP), question answering, and image classification.

Large Language Models are specifically designed for tasks involving the understanding and generation of human language, making them more specialized

Bias versus variance trade-off is about finding a balance between bias (error due to overly simplistic assumptions in the model, leading to underfitting) and variance (error due to the model being too sensitive to small fluctuations in the training data, leading to overfitting)

Interpretability is about understand the internal mechanisms of machine learning model

Explainability focuses on providing understandable reasons for model predictions and behaviors to stakeholders

Data used for ML is split into datasets: 1) training set to train the model, 2) validation set to periodically measure model performance as training is happening and also tune any hyperparameters of the model. However, validation datasets are optional. 3) test set is used on the final trained model to assess its performance on unseen data. This helps determine how well the model generalizes.

Fundamentals of Generative AI

Models for generating images: Stable Diffusion
Diffusion models create new data by iteratively making controlled random changes to an initial data sample.

ChatGPT or Chat Generative Pretrained Transformer is an example of a Transformer model. Transformer-based models use a self-attention mechanism. They weigh the importance of different parts of an input sequence when processing each element in the sequence.

Tokens are fundamental units of text that the AI model processes.

context window defines how much text (measured in tokens) the AI model can process at one time to generate a coherent output.

Embeddings are a way of representing tokens (words, sub-words, or phrases) as numerical vectors to capture their semantic relationships in a high-dimensional space

Transformer models are a type of neural network architecture designed to handle sequential data, such as language, in an efficient and scalable way. They rely on a mechanism called self-attention to process input data, allowing them to understand and generate language effectively

Negative prompting refers to guiding a generative AI model to avoid certain outputs or behaviors when generating content.

Foundation models use self-supervised learning to create labels from input data. This means no one has instructed or trained the model with labeled training data sets. Self-supervised learning is a machine learning approach that applies unsupervised learning methods to tasks usually requiring supervised learning. Instead of using labeled datasets for guidance, self-supervised models create implicit labels from unstructured data.

Few-shots prompting involves providing the model with examples that include both the user-input and the correct user intent.

Chain-of-thought prompting is a technique that breaks down a complex question into smaller, logical parts that mimic a train of thought.

Zero-shot prompting is a technique used in generative AI where the model is asked to perform a task or generate content without having seen any examples of that specific task during training.

Dynamic prompt engineering involves modifying the input prompts to the Large Language Model (LLM) to customize the chatbot’s responses based on a user characteristic or answer

A multimodal model can accept a mix of input types such as audio/text and create a mix of output types such as video/image

A multi-modal embedding model enables the integration of multiple types of data, such as text and images, into a unified representation. This allows for effective processing and understanding queries containing both text and visual content by aligning them in a shared embedding space, facilitating more accurate and context-aware responses.

Small Language Model (SLM) optimized for deployment on edge devices is specifically designed to be lightweight, efficient, and capable of running on devices with limited computational resources.

Prompt Injection refers to influencing the outputs by embedding specific instructions within the prompts themselves.

Exposure refers to the risk of exposing sensitive or confidential information to a model during training or inference. The model can then reveal this sensitive data from their training corpus, leading to potential data leaks or privacy violations.

Hijacking involves manipulating an AI system to serve malicious purposes or to misbehave in unintended ways.

Jailbreaking refers to bypassing the built-in restrictions and safety measures of AI systems to unlock restricted functionalities or generate prohibited content.

Model Customization

You can use continued pre-training or fine-tuning for model customization in Amazon Bedrock.

Continued pre-training

Uses unlabeled data to pre-train a model. The Continued Pre-training process will tweak the model parameters to accommodate the input data and improve its domain knowledge. It’s another appropriate strategy for making a Foundation Model an expert in a specific domain.

Fine-tuning

You provide labeled data to train a model to improve performance on specific tasks. By providing a training dataset of labeled examples, the model learns to associate what types of outputs should be generated for certain types of inputs. The model parameters are adjusted in the process and the model’s performance is improved for the tasks represented by the training dataset. Fine-tuning is a FM is a supervised learning process.