-
DL study – Jan 23, 2024
I was listening to the podcast between Lex and Ilya Sutskever and here are some takeaways: Designing DL model: other than installing inductive biases, the other main thing about DL architecture innovation lays in efficiency improvement. Ilya Sutskever mentioned in the Lex podcast that the success of Transformers come from the architecture being highly trainable,…
-
System 2 AI, DL – Jan 21, 2024
I have been challenging myself my notion that in order for a system to conduct deliberate thinking, its design has to have some certain inductive biases that underpin such an ability in human. But then I read that these days neuroscience’s inspirations on DL are more limited than I used to think, I started to…
-
DL theory study – Jan 20, 2024
DL by IG, YB and AC I have been reading the Introduction of the book and here are some notes:
-
DL implementation study – Jan 19, 2024
Building Micrograd Complete Micrograd notebook: https://github.com/gangfang/micrograd/blob/main/micrograd.ipynb
-
Can ChatGPT understand? A tiny case study – Jan 16, 2024
Can current machines understand? One observation that seems to be strongly showing ChatGPT doesn’t reason but merely imitate language formations is that it responds to more complex contextual information completely differently depending on the phrasing of the question, especially whether positivity or negativity is implied when a why question is asked. This shows that the…
-
DL implementation study, Jan 13, 2024
Building micrograd Notes Questions Why are gradient descent or its variants used to optimize parameters instead of solving the loss function analytically for its minima? AN: complex loss functions aren’t easily solvable analytically
-
DL implementation training – Jan 12, 2024
After a week of absence, I am finally back here writing. I was in contact with someone helpful and important who is already doing my dream research and is willing to guide me through my system 2 journey. And I was planning my next chapter of study, which is planned to last 3 months (or…
-
Subscribe
Subscribed
Already have a WordPress.com account? Log in now.
Gang Fang's Blog 