Gang Fang's Blog

This study is to help me understand this statement better:

The stochastic selection of just a few elements of content (that go into a thought) make GFlowNets a good candidate to implement the “consciousness priors”. In particular, the GWT bottleneck, when applied to such probabilistic inference, would enforce the inductive bias that the graph of dependencies between high-level concepts (that we can manipulate consciously and reason with) is very sparse

https://milayb.notion.site/The-GFlowNet-Tutorial-95434ef0e2d94c24aab90e69b30be9b3#208ee566b55048cda2c87fd5e0e93330

I think the first sentence talks about the sequential nature of GFlowNets operations facilitate the extraction process of a conscious state, described in 3.1 of the Consciousness Prior paper: once the encoder provides h_t, a sequential selection can occur among elements in h_t, which generates the compositional object (i.e., a conscious thought).

I don’t know yet if the second sentence involves GFlowNet, but I think it describes the GWT-inspired design where the joint distribution P(S) or P(h_t), approximating the world model, is represented by a sparse factor graph. The factor describes the dependencies between elements in h_t and the sparseness comes from the few h_t elements connected to a factor.

Where does the specification of “selection of just a few elements” come from? In the tutorial, a compositional object is built sequentially, but the number of steps isn’t specified.

TO_BE_ANSWERED.

Note: The consciousness prior and the GWT

The consciousness prior is inspired by the GWT bottleneck and an application of the bottleneck theory to System 2 AI construction.

Note: the inductive bias that…

This last sentence describes the sparse factor graph from a probabilistic modeling point of view. Each factor captures the strong dependency between a few variables, representing high-level concepts in a conscious thought built sequentially.

Note: world model building

We should think of this joint distribution as a very rough approximation of the world built by learning agents to help them plan, reason, imagine, etc.

https://arxiv.org/pdf/1709.08568.pdf%EF%BC%89%E3%80%82