AWS from Zero to Production — Phase 11, Part 1 of 3 · See the full series roadmap
Every engineering team is being asked the same question right now: “What’s our AI strategy?”
Behind the question is a lot of noise. Demos that look magical and fail on real data. Chatbots that confidently invent policies that don’t exist. Projects that spend months training a custom model when an API call would have done.
Engineers who understand the fundamentals cut through the noise. They know when a classic ML model is the right tool, when a foundation model is, how to tell if either is working — and why “just add our documents to the chatbot” is a real architecture called RAG, with real trade-offs.
This article gives you those fundamentals. No math degree required.
What you’ll walk away with
- 🔄 The machine learning lifecycle
- 🧑🏫 Supervised vs. unsupervised learning
- 🏋️ Training vs. inference
- 🧪 Feature engineering and model evaluation
- ⚖️ Responsible AI
- ✨ Generative AI and foundation models
- 🧭 Embeddings and vector databases
- 📚 Retrieval-augmented generation (RAG)
What machine learning actually is
Traditional software: you write the rules.
if order_total > 1000 and country != shipping_country and account_age_days < 7:
flag_for_review()
Machine learning: you provide examples, and an algorithm learns the rules.
100,000 past orders, each labeled "fraud" or "not fraud"
-> training -> a model that estimates fraud probability for new orders
ML is worth it when the rules are too complex to write by hand, change over time, or are hidden in lots of data. If you can write the rules, write the rules.
The ML lifecycle
1. Frame the problem what decision will this support? how will we measure success?
2. Collect data from the data platform (Phase 9)
3. Prepare data clean, label, split
4. Engineer features turn raw data into useful inputs
5. Train fit a model to training data
6. Evaluate measure on data the model has never seen
7. Deploy serve predictions (inference)
8. Monitor watch for drift, errors, and degraded quality
9. Retrain on fresh data — and repeat
Most of the work — and most of the failures — are in steps 1–4 and 8. The algorithm is often the easy part.
Engineer’s note: ML systems are software systems. They need versioning, CI/CD, monitoring, and rollbacks — the practice called MLOps. Everything from Phases 6–8 applies.
Supervised vs. unsupervised learning
Supervised learning
Learn from labeled examples — inputs paired with the right answer.
- Classification: predict a category. “Is this transaction fraud?” “Which support queue should this ticket go to?”
- Regression: predict a number. “How many orders tomorrow?” “What will this house sell for?”
The catch: you need labels, and good labels are often expensive.
Unsupervised learning
Find structure in unlabeled data.
- Clustering: group similar items. “What natural customer segments do we have?”
- Anomaly detection: find unusual items. “Which API calls look nothing like normal traffic?” — the idea behind CloudWatch anomaly detection and GuardDuty.
- Dimensionality reduction: compress many features into fewer.
And beyond
- Reinforcement learning: an agent learns by trial and reward — used in robotics, games, and to align language models.
- Self-supervised learning: a model learns from the data itself, like predicting the next word. This is how foundation models are trained.
Training vs. inference
- Training: the model learns from data. Compute-heavy, often on GPUs, done occasionally.
- Inference: the trained model makes predictions on new data. Done constantly, and must be fast, reliable, and cost-effective.
Training: historical data -> algorithm -> model artifact (stored in S3)
Inference: new input -> model -> prediction
Inference patterns:
- Real-time: an API returns a prediction in milliseconds — fraud scoring at checkout.
- Batch: score millions of records on a schedule — nightly churn predictions.
- Asynchronous: large inputs, results delivered later — document processing.
For most production systems, inference cost over time dwarfs training cost. Design for it.
Feature engineering
Features are the inputs a model learns from. Raw data rarely makes good features on its own:
Raw: order timestamp 2026-10-31T23:47:12Z
Features: hour_of_day=23, is_weekend=0, minutes_since_last_order=3,
orders_in_last_24h=9, account_age_days=2
Good features capture signal the model can use. Typical techniques:
- Aggregations over time windows — counts, averages, and trends
- Encoding categories as numbers
- Scaling numbers to similar ranges
- Handling missing values deliberately
- Domain knowledge — often the biggest win
A feature store — like Amazon SageMaker Feature Store — keeps features consistent between training and real-time inference.
Watch out — data leakage: If a feature contains information that wouldn’t be available at prediction time — like “refund issued” when predicting fraud at checkout — the model looks brilliant in testing and fails in production.
Model evaluation
Never judge a model on the data it trained on. Split the data:
- Training set — the model learns from it.
- Validation set — tune settings and compare models.
- Test set — the final, honest score, used once.
For time-based problems, split by time — train on the past, test on the future.
Metrics for classification
Accuracy is misleading when classes are imbalanced. If 0.5% of orders are fraud, a model that always says “not fraud” is 99.5% accurate — and useless.
- Precision: of the orders flagged as fraud, how many really were?
- Recall: of the real fraud, how much did we catch?
- F1 score: a balance of the two.
The right trade-off depends on the cost of each mistake: missing fraud vs. annoying a good customer.
Metrics for regression
- MAE (mean absolute error) and RMSE (root mean squared error) — how far off predictions are, on average.
Overfitting
A model that memorizes training data — including its noise — and performs badly on new data is overfit. Watch for a big gap between training and validation scores.
Responsible AI
Models make decisions that affect people. Responsible AI covers:
- Fairness: does the model perform differently — and unfairly — across groups?
- Explainability: can you explain why it made a decision?
- Privacy and security: is personal data protected in training data, prompts, and outputs?
- Safety: can it produce harmful content, or be manipulated into it?
- Veracity and robustness: is it accurate, and does it behave sensibly on unusual inputs?
- Transparency and governance: do users know when AI is involved? Who’s accountable?
On AWS: SageMaker Clarify detects bias and explains predictions; Bedrock Guardrails filters harmful content and PII (Part 2); and AI service cards document intended uses and limitations of AWS AI services.
Generative AI fundamentals
Generative AI creates new content — text, code, images, audio — instead of predicting a label or number.
Large language models (LLMs)
An LLM is trained on huge amounts of text to predict the next token — a word or piece of a word. At scale, that simple objective produces models that can summarize, translate, answer questions, write code, and follow instructions.
Key concepts:
- Prompt: the input — instructions, context, and the question.
- Tokens: what models read and write — and what you’re billed for.
- Context window: how many tokens the model can consider at once.
- Temperature: randomness — low for factual answers, higher for creative ones.
- Hallucination: a confident, fluent, wrong answer. LLMs generate plausible text, not verified facts.
Foundation models
A foundation model (FM) is a large model trained on broad data that can be adapted to many tasks — through prompting, retrieval, or fine-tuning — instead of training a model from scratch for each one.
Ways to customize, from cheapest to most expensive:
- Prompt engineering — clear instructions, examples, and output format.
- Retrieval-augmented generation (RAG) — give the model relevant facts at query time.
- Fine-tuning — further train the model on your examples to change its style or behavior.
- Continued pre-training or training from scratch — rarely justified outside specialist cases.
Rule of thumb: If the model lacks knowledge, use RAG. If it lacks a skill or style, consider fine-tuning. Try prompting first, either way.
Embeddings
An embedding is a list of numbers — a vector — that represents the meaning of text, an image, or other data. An embedding model produces it.
"How do I get a refund?" -> [0.012, -0.338, 0.761, ...] (e.g. 1,024 numbers)
"What's your return policy?" -> [0.019, -0.301, 0.744, ...] <- very close
"How do I reset my password?" -> [-0.552, 0.129, 0.087, ...] <- far away
Texts with similar meaning have vectors that are close together, even if they share no words. Closeness is measured with cosine similarity or distance.
That enables semantic search: find content by meaning, not just keywords.
Vector databases
A vector database stores embeddings and quickly finds the nearest ones to a query vector — usually with approximate nearest-neighbor (ANN) algorithms.
On AWS:
- Amazon OpenSearch Service and OpenSearch Serverless — vector search plus keyword search (hybrid).
- Amazon Aurora PostgreSQL and RDS for PostgreSQL with pgvector — vectors next to your relational data.
- Amazon S3 Vectors — low-cost vector storage and querying in S3.
- Amazon MemoryDB — in-memory vector search for very low latency.
- Amazon Neptune Analytics — vectors plus graphs.
Store metadata with each vector — source document, date, department, access level — so you can filter results.
Retrieval-augmented generation (RAG)
LLMs don’t know your company’s documents, and they can’t cite sources reliably. RAG fixes that by retrieving relevant content and giving it to the model with the question.
Ingestion (ahead of time)
Documents in S3
-> split into chunks (e.g. ~300 tokens with some overlap)
-> embed each chunk
-> store vectors + text + metadata in a vector database
Query (at request time)
User question
-> embed the question
-> retrieve the top-k most similar chunks (filtered by metadata/permissions)
-> build a prompt: instructions + retrieved chunks + question
-> the LLM generates an answer grounded in the chunks, with citations
Why RAG works
- Fresh knowledge — update documents, re-index, done. No retraining.
- Citations — users can check the source.
- Fewer hallucinations — the model answers from provided context, and can be told to say “I don’t know” otherwise.
- Access control — retrieve only what the user is allowed to see.
Where RAG goes wrong
- Bad chunking — splitting tables or sections in the middle.
- Poor retrieval — the right chunk isn’t in the top results. Most RAG quality problems are retrieval problems.
- Stale indexes — documents changed; vectors didn’t.
- Missing permission filters — the bot reveals documents the user shouldn’t see.
- No evaluation — nobody measures answer quality or groundedness.
You’ll build a RAG application in Part 3.
Choosing the right approach
Problem Start with
Predict a number/category from structured data Classic ML (SageMaker, or even SQL + rules)
Extract text/fields from documents AI services (Textract)
Sentiment, entities, PII in text AI services (Comprehend)
Answer questions about your documents RAG with a foundation model (Bedrock)
Draft, summarize, classify free text Foundation model + prompt engineering
Unique domain behavior at scale Fine-tuning, after prompting and RAG
Common mistakes
- Using ML where rules would do.
- Judging a model on accuracy with imbalanced classes.
- Data leakage that makes a model look great in testing.
- Treating LLM output as fact without grounding or verification.
- Fine-tuning to add knowledge that RAG would provide more cheaply.
- No evaluation set for generative AI — “it looked good in the demo.”
- Ignoring inference cost until the bill arrives.
Key takeaways
- ML learns rules from examples; the lifecycle — data, features, evaluation, monitoring — matters more than the algorithm.
- Supervised learning needs labels; unsupervised learning finds structure.
- Training is occasional; inference is constant — design for inference cost and reliability.
- Evaluate honestly with held-out data and the right metrics.
- Foundation models are customized with prompting, RAG, and fine-tuning — in that order.
- Embeddings capture meaning; vector databases find similar content; RAG grounds answers in your data.
Quick self-check
Test yourself before moving on:
- Predicting tomorrow’s order count: classification or regression?
- Why is accuracy a poor metric for fraud detection?
- Your chatbot doesn’t know your company’s latest refund policy. RAG or fine-tuning?
- What does an embedding represent?
Answers
- Regression — it predicts a number.
- Fraud is rare; a model that never flags fraud would still be highly “accurate.” Use precision and recall.
- RAG — it’s missing knowledge, which changes over time.
- The meaning of content, as a vector — similar meanings are close together.
What’s next
In Part 2, we tour the AWS AI toolbox: Amazon Bedrock and model selection, Knowledge Bases, Agents, and Guardrails, SageMaker for custom models, and ready-made AI services like Textract, Comprehend, Rekognition, Transcribe, and Translate.
What’s the most useful — or most overhyped — AI use case you’ve seen at work? Tell me in the comments.