Category: AI
-
Weekly DL study note: GFlowNet Code Tutorial (completed)
Completed code: https://github.com/gangfang/littlegfn/blob/main/face_generator.ipynb Pre-requisites Flow Networks (Nothing about training yet) IMPORTANT – MAIN IDEA OF GFN The main idea behind GFlowNet is to interpret the DAG as a flow network, and to think of each edge as a pipe through which some amount of water, or particles, flows. We then want to find a flow where, (a) flow is…
-
Weekly study note: GFlowNet Code Tutorial 2022
Flow Networks (Nothing about training yet) IMPORTANT – MAIN IDEA OF GFN The main idea behind GFlowNet is to interpret the DAG as a flow network, and to think of each edge as a pipe through which some amount of water, or particles, flows. We then want to find a flow where, (a) flow is preserved, (b) the flow…
-
Experiments with a 2M-param transformer
Jupyter notebook: https://github.com/gangfang/nanogpt/blob/main/gpt_dev.ipynb
-
Autoencoder, contrarian thought on AGI and GenAI’s impacts on Software engineering, Interpretability
Autoencoder: Contrarian thought on AGI It’s possible that AGI will never arrive. AI practitioners complain about benchmarks being pushed further and further whenever they make progress. This situation makes sense when viewed from the lens of Bayesian framework. The prior changed, which leads to changes in expectations. Inferring from this, I speculate that it is…
-
Weekly DL study notes: building GPT from stretch, part 2
Parallelization: By removing sequential dependencies, Transformers could be trained much more efficiently on parallel hardware like GPUs.
-
Weekly DL study notes: building GPT from scratch
Updated on Jun 14, 2024 Residual Networks (ResNets) Attention
-
DL weekly study notes: building a wavenet
Code reproduction of Andrej Karpathy’s “building makemore part 5” lecture: https://github.com/gangfang/makemore/blob/main/makemore_part5.ipynb Study notes:
-
DL weekly study notes: manual backprop i.e., w/o loss.backward()
Code reproduction of Andrej Karpathy’s “building makemore part 4” lecture: https://github.com/gangfang/makemore/blob/main/makemore_part4_manual_backprop.ipynb
Gang Fang's Blog 