-
Seeds for research
I adopted the concept of seeds from the book, the creative act by Rick Rubin here to document all interesting ideas that I want to pursue scientifically: (they will be organized into blog posts) Most seeds above come from https://yoshuabengio.org/2023/03/21/scaling-in-the-service-of-reasoning-model-based-ml/ -> GFlowNet is a manifestation of all these ideas. More seeds
-
Thoughts on “Ideas”
Ideas are the core concepts that both science and art can relate to and share, and they are the ultimate proponents of human progress. The above statement might sound like a cliché, but it becomes powerful when put into action: instead of passively consuming information presented to me by the media (e.g., news), I actively…
-
Inductive Biases
“”” Inductive bias refers to the set of assumptions that a learner (human or machine) makes to predict outputs for previously unseen inputs when trying to learn from a given set of training examples. In other words, it’s the bias or set of preferences a learning algorithm has when inferring a hypothesis or function from…
-
More concepts
“Convex Combination” The term “convex” in mathematics usually refers to a set of points that, if you take any two points within the set and draw a line segment between them, every point on that line segment also belongs to the set. However, in the context of the paragraph, “convex” refers to a specific type…
-
Reductionism Can’t Address Complex Systems
A colleague once said that drones are not innovative because they are just four motors and some plastic put together, arguing that these are existing components. However, I don’t believe that this reductionist argument fully explains the true nature of these devices. This particular argument stuck with me, as I had recently been contemplating a…
-
Being a Dancer Doesn’t Guarantee Originality
Written originally in English. 中文译文在英文版后。 A long time ago, I read a YouTube comment on a popping competition video along these lines: the quality of dance has deteriorated compared to the skills of the OGs who invented everything in the 70s and 80s. What makes this argument notable is our usual assumption that things are…
-
Sitting in a pocket in freestyling
I found the oldest footage of MT Pop on YouTube. This is 13 years old. It shows why MT Pop is able to sit in a pocket of a style or moves and seemingly create unlimited number of variations – he practiced using one style throughout an entire song, and the song is over 4…
-
Current AI as Artificial Intuition
Looking at state-of-the-art (SOTA) artificial intelligence (AI), which is based on deep learning (DL), through the lens of dual process theory, it’s easy to see that what SOTA AI truly accomplishes is the intuitive aspect of intelligence (in the broadest sense). This is akin to our ability to automatically coordinate our bodies when we run…
-
Distributed representation
Distributed representation is at the core of why ANNs work. What is distributed representation in ML? Distributed representation in machine learning refers to a way of representing data where each input is described by numerous elements or features, and each feature can contribute to the representation of many inputs. In other words, the information is…
-
Compositionality
It helps IID and OOD generalization because lower level features in smaller quantities can be combined in different way to generalize to something out of distribution of the training data. It has forms such as: And the Bengio’s goal is to introduce more forms of compositionality.
-
Subscribe
Subscribed
Already have a WordPress.com account? Log in now.
Gang Fang's Blog 