Transformer Architecture with TensorFlow

This repository contains the programming assignment from Week 4 of Course 5: Sequence Models in the Deep Learning Specialization by Andrew Ng on Coursera.

Overview

In this assignment, the Transformer model is built from scratch using TensorFlow and Keras layers.
The goal is to understand and implement the key components that power modern sequence models, particularly in the context of natural language processing (NLP).

Key concepts covered:

• Scaled dot-product attention
• Multi-head attention mechanism
• Positional encoding for sequence inputs
• Encoder and decoder blocks
• Full Transformer model integration

This assignment provides hands-on experience with the architecture that underlies models such as BERT, GPT, and T5.

Contents

• ScaledDotProductAttention – Implementation of scaled dot-product attention
• MultiHeadAttention – Construction of multi-head attention layers
• PositionalEncoding – Adds positional information to input embeddings
• EncoderLayer and DecoderLayer – Building blocks of the Transformer
• Encoder and Decoder – Stacked encoder and decoder components
• Transformer – Complete Transformer model for sequence-to-sequence tasks
• Jupyter Notebook – Full walkthrough of architecture, training, and evaluation

Technologies Used

• Python 3
• TensorFlow 2.x / Keras
• NumPy

Course Information

• Course: Sequence Models (Course 5 of 5)
• Specialization: Deep Learning Specialization
• Instructor: Andrew Ng
• Platform: Coursera
• Institution: DeepLearning.AI

License

This repository is based on educational material provided by DeepLearning.AI through Coursera.
It is intended for educational and personal use only.

---