Training Neural Networks with Backpropagation and Gradient Descent

Business Scenario

Welcome!

In Lab 1 you built the SmartCart ANN (Input -> Hidden ReLU -> Hidden ReLU -> Output Softmax) in Keras and called .fit() to train it — the loop worked, but forward pass, loss, backprop, and weight update were all hidden inside one method.

The ML lead wants proof you understand what .fit() does each epoch. Task: rebuild the training loop step-by-step in Keras/TensorFlow on the same SmartCart product-category network and dataset, using tf.GradientTape instead of .fit() so every step is visible in your own code. Show loss falling epoch by epoch, and explain each step.

Pre-Lab Preparation

Topic: Backpropagation and Gradient Descent

1) Loss Function
2) Chain Rule

3) Gradients, Gradient Descent
4) Backpropagation Flow
5) Epochs & Convergence

 

Git Pull

git pull origin branchName

Setup

1

Task 1: Reload Lab 1's Network and Dataset

import numpy as np, pandas as pd
import tensorflow as tf
from tensorflow import keras
from keras import Sequential
from keras.layers import Dense
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler, LabelEncoder
from sklearn.metrics import accuracy_score
import matplotlib.pyplot as plt

np.random.seed(42)
tf.random.set_seed(42)

Load Dataset

2

Dataset : 

df = pd.read_csv("smartcart_dataset.csv")
FEATURES = ["price", "weight_kg","rating", "discount_pct", "description_length"]

TARGET = "category"
df.head()

Output

Preprocessing

3

X = df[FEATURES].values
y_raw = df[TARGET].values

label_encoder = LabelEncoder()            # Encode target labels
y = label_encoder.fit_transform(y_raw)
class_names = label_encoder.classes_

X = df[FEATURES].values
y_raw = df[TARGET].values

label_encoder = LabelEncoder()                      # Encode target labels
y = label_encoder.fit_transform(y_raw)
class_names = label_encoder.classes_

X_train, X_test, y_train, y_test =                  # Split data into training & testing
    train_test_split(                                   
    X, y,test_size=0.2, random_state=42, 
    stratify=y)

scaler = StandardScaler()                           # Scale input features
X_train_scaled = scaler.fit_transform(X_train)     
X_test_scaled = scaler.transform(X_test)

X_train_tf = tf.convert_to_tensor(X_train_scaled,   # Convert data into Tensors
dtype=tf.float32)
y_train_tf = tf.convert_to_tensor(y_train, 
dtype=tf.int32)

X_test_tf = tf.convert_to_tensor(X_test_scaled, 
dtype=tf.float32)
y_test_tf = tf.convert_to_tensor(y_test, 
dtype=tf.int32)

Build Neural Network

4

model = Sequential()

model.add(Dense(32, activation="relu", input_dim=X_train_scaled.shape[1]))
model.add(Dense(16, activation="relu"))
model.add(Dense(3, activation="softmax"))

model.summary()

Forward Pass

1

Task 2 : Manual Neural Network Training

def forward(model, X):
    return model(X, training=True)

Output

Back Propagation

3

def backward(model, X, y_true):
    with tf.GradientTape() as tape:
        y_pred = forward(model, X)
        loss = compute_loss(y_true, y_pred)

    gradients = tape.gradient(loss, model.trainable_variables)
    return loss, gradients
optimizer = keras.optimizers.SGD(learning_rate=0.1)

def update(model, gradients, optimizer):
    optimizer.apply_gradients(zip(gradients, model.trainable_variables))

Weight Update

4

Loss Calculation

2

loss_fn = keras.losses.SparseCategoricalCrossentropy()
def compute_loss(y_true, y_pred):
    return loss_fn(y_true, y_pred)

Training Loop and Loss Tracking

5

loss_history = []
for epoch in range(500):
    loss, gradients = backward(model, X_train_tf, y_train_tf)
    update(model, gradients, optimizer)

    loss_history.append(float(loss))

    if epoch % 50 == 0:
        print(f"Epoch {epoch:4d} │ Loss: {float(loss):.4f}")
plt.plot(loss_history)
plt.xlabel("Epoch")
plt.ylabel("Sparse categorical cross-entropy loss")
plt.title("SmartCart ANN (Keras) — manual GradientTape training loop loss")
plt.show()

Output

Model Evaluation

6

test_probs = model(X_test_tf, training=False)       # no training here

y_pred = np.argmax(test_probs.numpy(), axis=1)      # highest-probability class

print("Test accuracy:", round(accuracy_score(y_test, y_pred), 4))

Output

 

Great job!

You have successfully rebuilt the Lab 1 network’s training loop step-by-step in Keras/TensorFlow without .fit(), separated the forward pass, loss, backpropagation, and gradient descent using tf.GradientTape, tracked loss reduction over 500 epochs, and verified comparable test accuracy on the same SmartCart dataset.

Checkpoint

   Git Push

git push origin branchName

Next-Lab Preparation

Topic : NLP Foundation

1) Introduction to NLP
2) Text Processing - (Cleaning, tokenization, lemmatization)
3) Vectorization - Bag of words(BoW) / TF-IDF
4) Word Embeddings