AWS Learning Journey

AI Without the Hype: ML and Generative AI Fundamentals for Engineers

The machine learning lifecycle, supervised and unsupervised learning, training and inference, features, evaluation, responsible AI — and the generative AI essentials: foundation models, embeddings, vector databases, and retrieval-augmented generation.

11 min read · 2013 words
Machine LearningGenerative AIAWSArtificial IntelligenceLLM
AI Without the Hype: ML and Generative AI Fundamentals for Engineers cover image

AWS from Zero to Production — Phase 11, Part 1 of 3 · See the full series roadmap


Every engineering team is being asked the same question right now: “What’s our AI strategy?”

Behind the question is a lot of noise. Demos that look magical and fail on real data. Chatbots that confidently invent policies that don’t exist. Projects that spend months training a custom model when an API call would have done.

Engineers who understand the fundamentals cut through the noise. They know when a classic ML model is the right tool, when a foundation model is, how to tell if either is working — and why “just add our documents to the chatbot” is a real architecture called RAG, with real trade-offs.

This article gives you those fundamentals. No math degree required.

What you’ll walk away with


What machine learning actually is

Traditional software: you write the rules.

if order_total > 1000 and country != shipping_country and account_age_days < 7:
    flag_for_review()

Machine learning: you provide examples, and an algorithm learns the rules.

100,000 past orders, each labeled "fraud" or "not fraud"
  -> training -> a model that estimates fraud probability for new orders

ML is worth it when the rules are too complex to write by hand, change over time, or are hidden in lots of data. If you can write the rules, write the rules.


The ML lifecycle

1. Frame the problem     what decision will this support? how will we measure success?
2. Collect data          from the data platform (Phase 9)
3. Prepare data          clean, label, split
4. Engineer features     turn raw data into useful inputs
5. Train                 fit a model to training data
6. Evaluate              measure on data the model has never seen
7. Deploy                serve predictions (inference)
8. Monitor               watch for drift, errors, and degraded quality
9. Retrain               on fresh data — and repeat

Most of the work — and most of the failures — are in steps 1–4 and 8. The algorithm is often the easy part.

Engineer’s note: ML systems are software systems. They need versioning, CI/CD, monitoring, and rollbacks — the practice called MLOps. Everything from Phases 6–8 applies.


Supervised vs. unsupervised learning

Supervised learning

Learn from labeled examples — inputs paired with the right answer.

The catch: you need labels, and good labels are often expensive.

Unsupervised learning

Find structure in unlabeled data.

And beyond


Training vs. inference

Training:   historical data -> algorithm -> model artifact (stored in S3)
Inference:  new input -> model -> prediction

Inference patterns:

For most production systems, inference cost over time dwarfs training cost. Design for it.


Feature engineering

Features are the inputs a model learns from. Raw data rarely makes good features on its own:

Raw:       order timestamp 2026-10-31T23:47:12Z
Features:  hour_of_day=23, is_weekend=0, minutes_since_last_order=3,
           orders_in_last_24h=9, account_age_days=2

Good features capture signal the model can use. Typical techniques:

A feature store — like Amazon SageMaker Feature Store — keeps features consistent between training and real-time inference.

Watch out — data leakage: If a feature contains information that wouldn’t be available at prediction time — like “refund issued” when predicting fraud at checkout — the model looks brilliant in testing and fails in production.


Model evaluation

Never judge a model on the data it trained on. Split the data:

For time-based problems, split by time — train on the past, test on the future.

Metrics for classification

Accuracy is misleading when classes are imbalanced. If 0.5% of orders are fraud, a model that always says “not fraud” is 99.5% accurate — and useless.

The right trade-off depends on the cost of each mistake: missing fraud vs. annoying a good customer.

Metrics for regression

Overfitting

A model that memorizes training data — including its noise — and performs badly on new data is overfit. Watch for a big gap between training and validation scores.


Responsible AI

Models make decisions that affect people. Responsible AI covers:

On AWS: SageMaker Clarify detects bias and explains predictions; Bedrock Guardrails filters harmful content and PII (Part 2); and AI service cards document intended uses and limitations of AWS AI services.


Generative AI fundamentals

Generative AI creates new content — text, code, images, audio — instead of predicting a label or number.

Large language models (LLMs)

An LLM is trained on huge amounts of text to predict the next token — a word or piece of a word. At scale, that simple objective produces models that can summarize, translate, answer questions, write code, and follow instructions.

Key concepts:

Foundation models

A foundation model (FM) is a large model trained on broad data that can be adapted to many tasks — through prompting, retrieval, or fine-tuning — instead of training a model from scratch for each one.

Ways to customize, from cheapest to most expensive:

  1. Prompt engineering — clear instructions, examples, and output format.
  2. Retrieval-augmented generation (RAG) — give the model relevant facts at query time.
  3. Fine-tuning — further train the model on your examples to change its style or behavior.
  4. Continued pre-training or training from scratch — rarely justified outside specialist cases.

Rule of thumb: If the model lacks knowledge, use RAG. If it lacks a skill or style, consider fine-tuning. Try prompting first, either way.


Embeddings

An embedding is a list of numbers — a vector — that represents the meaning of text, an image, or other data. An embedding model produces it.

"How do I get a refund?"          -> [0.012, -0.338, 0.761, ...]  (e.g. 1,024 numbers)
"What's your return policy?"      -> [0.019, -0.301, 0.744, ...]  <- very close
"How do I reset my password?"     -> [-0.552, 0.129, 0.087, ...]  <- far away

Texts with similar meaning have vectors that are close together, even if they share no words. Closeness is measured with cosine similarity or distance.

That enables semantic search: find content by meaning, not just keywords.


Vector databases

A vector database stores embeddings and quickly finds the nearest ones to a query vector — usually with approximate nearest-neighbor (ANN) algorithms.

On AWS:

Store metadata with each vector — source document, date, department, access level — so you can filter results.


Retrieval-augmented generation (RAG)

LLMs don’t know your company’s documents, and they can’t cite sources reliably. RAG fixes that by retrieving relevant content and giving it to the model with the question.

Ingestion (ahead of time)

Documents in S3
  -> split into chunks (e.g. ~300 tokens with some overlap)
  -> embed each chunk
  -> store vectors + text + metadata in a vector database

Query (at request time)

User question
  -> embed the question
  -> retrieve the top-k most similar chunks (filtered by metadata/permissions)
  -> build a prompt: instructions + retrieved chunks + question
  -> the LLM generates an answer grounded in the chunks, with citations

Why RAG works

Where RAG goes wrong

You’ll build a RAG application in Part 3.


Choosing the right approach

Problem                                          Start with
Predict a number/category from structured data   Classic ML (SageMaker, or even SQL + rules)
Extract text/fields from documents               AI services (Textract)
Sentiment, entities, PII in text                 AI services (Comprehend)
Answer questions about your documents            RAG with a foundation model (Bedrock)
Draft, summarize, classify free text             Foundation model + prompt engineering
Unique domain behavior at scale                  Fine-tuning, after prompting and RAG

Common mistakes


Key takeaways


Quick self-check

Test yourself before moving on:

  1. Predicting tomorrow’s order count: classification or regression?
  2. Why is accuracy a poor metric for fraud detection?
  3. Your chatbot doesn’t know your company’s latest refund policy. RAG or fine-tuning?
  4. What does an embedding represent?

Answers

  1. Regression — it predicts a number.
  2. Fraud is rare; a model that never flags fraud would still be highly “accurate.” Use precision and recall.
  3. RAG — it’s missing knowledge, which changes over time.
  4. The meaning of content, as a vector — similar meanings are close together.

What’s next

In Part 2, we tour the AWS AI toolbox: Amazon Bedrock and model selection, Knowledge Bases, Agents, and Guardrails, SageMaker for custom models, and ready-made AI services like Textract, Comprehend, Rekognition, Transcribe, and Translate.

What’s the most useful — or most overhyped — AI use case you’ve seen at work? Tell me in the comments.


Share this article

LinkedIn post draft