American Express - Default Prediction

As team leader, I controlled pace, assigned tasks, and organized brainstorming...
Keywords: Machine Learning · Stacking & Blending · Boosting algorithms

[Competition Website: American Express ]
[Code Repository: View my code here ]

Analysis of thyroid disease datasets

- Explore, cleanse and process thyroid disease data...
Keywords: R language · Shiny · KNN imputation · PCA

[Project Requirements: The Thyroid Disease Data Set ]
[Code Repository: View my code here ]

PyTorch Vision Transformer — Ants vs. Bees

The project fine-tunes a Vision Transformer for a two-way classification task (bees vs. ants), standing on top of the open-source ViT-pytorch weights and a lightweight PyTorch training stack. It emphasizes rapid experimentation with data pipelines, attention heads, and inference tooling so that the ViT backbone can be easily re-used for other small datasets. Current utilities cover loading pretrained models, evaluating them on ImageNet or custom data, and quickly fine-tuning on bespoke datasets, while upcoming work aims at full from-scratch training and ONNX export.
Keywords: Visual Transformers · PyTorch · Transfer Learning · Attention

[Project Repository: Click here ]
[Dataset: Ants & Bees on Kaggle ]
[Related Documents: Google Drive folder (brief + dashboard) ]
[Short Report: Download here ]
[Training Dashboard: View metrics ]

Current focus

• • Use vectorized L2 distance in attention for the discriminator.
• • Explore overlapping image patches and DiffAugment.
• • Prototype Self-modulated LayerNorm (SLN) and implicit patch generators.
• • Stabilize training with EMA, Balanced Consistency Regularization (bCR), Improved Spectral Normalization (ISN), and Equalized Learning Rate.
• • Experiment with weight modulation to better align ViT adapters with small datasets.

Under construction [ing]

Under construction [ing]
Keywords: Under construction [ing]

Under construction [ing]

An attempt of Vit model for image classification task

Bi-directional classification task (i.e., between bees and ants) ...
Keywords: Visual Transformers (ViT) · PyTorch · Image Processing

[Report: A brief report ]
[Dataset: Ants & Bees ]
[Code Repository: View my code here ]

LLM from Scratch (In Progress)

- Building a compact yet complete transformer to understand every layer from tokenizer and embeddings to RMSNorm, attention, and causal decoding.
- Authoring training, evaluation, and inference loops purely in PyTorch so experimental components (attention variants, optimizers) can be swapped rapidly.
Keywords: Large Language Model · PyTorch · Transformer · Tokenizer design

[Code Repository: Click here ]