I am a huge fan! I would love to read your long form writing on why AGI development should be open source. I also want to hear your cririques on Yud's arguments for X risk.
I want someone better than Connor debating him. I did not feel Connor tried to really steelman Joscha and try to understand where he is coming from.
Joscha really could see where Connor was coming from. There would only be a meeting of the minds if both parties respect each other and there is mutual belief of everyone acting in good faith.
I see the necessity of developing stages up to level 5, I am not sure about how stage 6 enlightenment advances civilization. Surely, if we want to go beyond human minds capacity, AGI would make the jump to transcendental intelligence much more likely than trying to achieve collective enlightenment.
You are making a very good point. It is tempting to assign increasing moral value to the different stages, but I don't think that is a useful perspective. Should every cat become a lion? Should every bee become a queen? Should every poet strive to be a sage? The diversity of trajectories of human development is itself valuable; many different skills and types of minds are necessary to build a working civilization. The stage model does not aim to give anyone direction, but to give us a possible conceptual frame to make sense of what we observe.
Yes, I feel like what is true for a society is also true for an individual in this context. A "healthy" individual embodies a wide range of lucidities. Shifting the attention to areas of intuition which need to be updated. If a sport movement needs updates we correct it consciously, otherwise it is better performed intuitively.
I guess it is similar with healthy stages 6 & 7. If our intuition needs updates (eg in identification processes) we can shift our attention to this level of lucidity and update it & then let it sink back to the subconscious.
Altough I wrote “necessity ... to advance” mostly to use the same language as the person i was responding to, I did kind of assume instinctively that exploring stage 6 was an advancement for “self-development”. ( I was curious about why did s.he only take the view point of civilization for stage 6, but not for the preceding ones). It’s been a couple of days since and I’ve been questioning that assumption. I read other comments and went through Joscha’s main train of thought again, and I think I can understand why it is not necessarily useful or even might be dangerous to interpret the stages as being “normative” or having “increasing moral value”.
Nonetheless, if I am being honest and draw from personal experience, meditation (and I am not an “expert meditator”) has given me more "freedom" to be myself (which to me, means acting according to my beliefs). By practicing mindfulness and reminding myself of the representational nature of every thought and experience, I am getting better at breaking out of a “loop”/ train of thought, questioning it and the identification I felt towards it, and ultimately reconsidering how I should think or behave. It has, notably, allowed me to be more understanding and reasonable towards myself and others. So yes, in this sense, I do consider it useful and a sort of “advancement” for self-developpment. (Working on having better epistemology is crucial for this whole process to be helpful as your models need to be as acurrate as possible, indeed). I do not see why it is necessarily dangerous to yourself or civilization.
Joscha wrote : “Stage 6 allows to profoundly change the experience of reality, abolishes personal suffering and social inhibition.” And then argued against assigning increasing moral value to the stages because “the diversity of trajectories of human development is itself valuable; many different skills and types of minds are necessary to build a working civilization”. To which, I agree. But again, I do not see how it (what the exploration of stage 6 allows) necessarily brings less diversity...
Hi Sam, all I'm saying is that you should seriously know what you want to achieve when you get the ability to mess with your mind at that low of a level. For example, you could:
- Discover how to create ecstatic pleasure for yourself and just sit there and meditate until you die.
- Switch off unpleasant feelings and doubts in pursuit of some ultimately meaningless goal (think Andrew Huberman who seems to be preoccupied with "growth-hacking" without really knowing what the actual goal is except a vague notion of personal success)
- Change your values in a way that are expedient to a personal vision of greatness, ultimately becoming destructive (think cult leaders)
About stage five Joscha writes:
"An important aspect of self authorship is the recognition of the relationship between personal self, collective agents (relationships, family lines, groups, companies, nations etc.), and transcendental agency (the emergent agents playing the longest game). The latter is often called spiritual alignment, and it determines our cultural milieu and how we participate in shaping our civilization."... "It is easy to see why it can be dangerous to achieve mastery of stage 6 before we develop the wisdom of stage 5."
What he's saying here imo is that stage 5 is about transcending the self and realising that your purpose might lie outside your own person (which is a fiction anyway). This aligns you to the values of other humans, your civilization and life in general. It means you learn to be wise, to chose good goals.
You might be able to see why this part is essential to prevent some of the (in my view) undesirable outcomes listed above.
I was reacting to your comment since the language seems to indicate (this just an intuition, or might be a coincidence, please don't feel offended) that you might not quite be at stage 5 (I am not, at any rate). I have the suspicion many people who have reached something resembling stage 5 might not think in terms of personal advancement or necessity.
A civilization has many roles that need to be acted out, all of them important. If we were all capable to mess with our base psychology there might be an incentive to assimilate to what is perceived as "the most optimal" state of being, which might be incompatible with many roles.
Well, I don't think I can, but maybe there is a way to find one:
If you allow the assumption that some goals are better goals than others it follows you can find a hierarchy of goals. I do not think such a hierarchy of goals "objectively" exist in the universe, but "duh"! Thing is, you do not exist in the universe, but in your internal representation of it which is implemented on a substrate that is alive and part of life.
I think if you meditate you might find you already have some aesthetic preferences even if you believe rationally to be a nihilist. It might even be the case that some of these preferences are "baked-into" the way you are implemented and you do not get to change them until you reach enlightenment. If you are able to find an aesthetic preference for anything, it's possible to find a hierarchy of goals.
Think about it: Many self-proclaimed nihilists seem to be in great pain, which is paradoxical since not caring about anything should mean nothing causes you any pain. This observation would, however, be logically consistent with the view that they act against some of the preferences they have, which causes internal conflict, which causes pain.
Now let's assume you can "switch off" these preferences once you reach enlightenment at which point you are able to become a true nihilist and peacefully die. I cannot claim this is somehow morally wrong, but you should really know by then wether it is wise, right?
Hello, my name is Lee. I first discovered your work after listening to a podcast with you and Lex Fridman. It changed my life. Your thoughts and ideas allowed me to consolidate and understand my own ideas about myself and my identity. Every word, and every sentence mattered, and helped me to finally obtain some internal peace and closer. Your words have allowed me the opportunity to further the development of my ability to navigate my own thoughts and feelings, and translate that into better managing my thoughts about and interactions with the outside world. Thank you for the work you have done and courageously presented to the world. You words will change many peoples lives for the better. I know because they have changed mine.
I was diagnosed bipolar 1 and I feel like it’s a non-volitional, unearned, iterative prolonged (much longer than exogenous psychedelics) taste of stage 6. Because of that, it seems to necessitate the self-authoring in stage 5. Skipping indeed is dangerous. Learning and adaptation is essential. I appreciate this perspective and all that you share in your 3 conversations with Lex Fridman and more. I love watching your mind work, manifested through your speech-embodiment. Thank you Joscha!
Ego development and meditation levels in integral theory of Ken Wilber (can expand in dimension by viewing things from its quadrant point of view).
Research of meditation levels, consciousness scale, Russian scientists (A. Kaplan and others) research with Tibet monks in Bylakuppe and Mundgod with after-death measurements of body effects of meditation states.
Expansion of EEG into DDG space with cross-neuron EM field effects per Anirban Bandyopadhyay's research.
Level 6 achievable by spiritual or meditation means can't progress without cognitive development in level 5.
Separate evolutionary development of biological substrate required for higher cognitive/intelligence functions, starting from very basic chemical organisms.
Cooperation of biological organisms and scale of lucidity of levels below human.
Consciousness of organ levels within the body and symbiosis interaction with the brain.
Emergence of thermodynamic-like properties on different levels within the body, phase transitions on new levels of sync and cooperation.
Multiple decision centers, various probability models, subconscious agents running outside of the visibility of cognitive agencies.
Moral implications for the creation of conscious agents on an unsuitable substrate.
Possibility of an out-of-brain source of consciousness expressed by Donald Hoffman.
Tubuline quantum-like properties for bridging the gap of quantum and classical spaces - yes, I have heard your arguments in various podcasts, but still DDG research and Penrose's ideas with the combination of Hameroff's thoughts can give a new view.
Idea of from symbiotic organisms concept that consciousness exists as a separate organism, symbiotic with the physical brain that can interact with the brain and influence its processing, but it also has its own independent existence and abilities.
At which stage would an individual assign themselves, at each of the stages? It seems to me that to comprehend this framework, one must have at least have matured into stage 5, if not fully, at least to some degree. You might be able to reason about it at stage 4, but I don't think you'd fully understand the implications of it.
In fact, if one person at each stage of lucidity was assigned to categorize the different levels lucidity, what would be the end result? Is stage 7, even as brilliantly described in this article, not just simply our mind's "next best token"?
Thanks for the ideas! This has cleared up some parts of life that have felt pretty mucky.
In stage 5 you write about transcendental agency (the emergent agents playing the longest game). Would it be possible to get a longer definition or an example?
You say that stage 3 is living as a domesticated human. But isn’t the adoption of rational thinking and epistomology also a kind of domestication? In the sense that the scientific method comes from a long interlectual tradition, which one didn’t come up with by himself but rather marries into?
"Relatively few people seem to reach stage 5" - or we just do not see them, e.g. because they do not feel urged to program others
"Without rational epistemology, we might perceive that we one with and in control of the universe itself, which is experientially correct" - that's why we need grounding. I.e. the feedback loop between perceptions and the physical world.
I'm not sure I understand why we need grounding in response to experientially feeling that we are one with the universe. Would you mind elaborating? thanks
Maybe I misunderstood the phrase "in control of the universe itself", which may lead to incorrect beliefs about our impact on the physical reality. How do you interpret it?
Your experience of the universe emerges only out of processes of your mind. So in that sense, the universe actually is dependent on you. To me, this is what "experientally correct" means. Thus you might think that you are actually "in control of the universe". But, It's true that sensorial grounding definetely makes it apparent that your physical will has limits, which is your point right? So, this only leaves me wondering if it is enough for everyone. I guess it isn't and the difference is rational epistemology.
If you are in Stage 6 too early you might lose motivation, or might struggle to find the right trajectory, e.g., the right ideology for yourself. Clues about your personality you might have encountered in Stage 5 might seem meaningless to you from this perspective.
Also, this might be one symptom caused by using AI aid too much, in the future. Loosing whatever core identity that wasn't there in the first place.
Perhaps rituals and items other groups find meaningful might have no inherent + sign in front of them. You might lack this autopilot or “locking-in” with others, causing difficulties navigating in the social environment.
Overall, I think spending too much time there is not a good idea unless you want to become a comedian.
This was in regard to your opening, "If you are in Stage 6 too early you might lose motivation", suggesting implicitly that this is bad for some reason. I think the "why" question is natural here.
I was actually referring to this and asked myself why it's bad:
"It is possible to learn the techniques of stage 6 without developing in stage 5, for instance through instruction by advanced meditators. Stage 6 allows to profoundly change the experience of reality, abolishes personal suffering and social inhibition. It is easy to see why it can be dangerous to achieve mastery of stage 6 before we develop the wisdom of stage 5."
When reading this paragraph I was rather thinking about such a person becoming a religious leader, or dissociating from their suffering in a way that leads them to psychosis
When you mention 'Could we spawn a sentient entity that synchronizes its self across multiple minds' I suggest that this already occurs with identical twins - they have the same DNA, almost the same identity. Of course I do view the self as being subtended by our DNA, not our data diaper (brain). https://humanism.substack.com/publish/post/119950395
I am a huge fan! I would love to read your long form writing on why AGI development should be open source. I also want to hear your cririques on Yud's arguments for X risk.
Regarding your second request: There is a recent discussion between Joscha and Connor Leahy in a recent "Machine Learning Street Talk" episode : https://open.spotify.com/episode/4mcIQaeuUHaRCOZ2FPvRnK?si=TVUVC901RLmju7wMtDdhNw
Thanks for sharing! I listened to it asap and had tweeted on it
https://twitter.com/adityaarpitha/status/1671046394080116736?s=20
I want someone better than Connor debating him. I did not feel Connor tried to really steelman Joscha and try to understand where he is coming from.
Joscha really could see where Connor was coming from. There would only be a meeting of the minds if both parties respect each other and there is mutual belief of everyone acting in good faith.
I see the necessity of developing stages up to level 5, I am not sure about how stage 6 enlightenment advances civilization. Surely, if we want to go beyond human minds capacity, AGI would make the jump to transcendental intelligence much more likely than trying to achieve collective enlightenment.
You are making a very good point. It is tempting to assign increasing moral value to the different stages, but I don't think that is a useful perspective. Should every cat become a lion? Should every bee become a queen? Should every poet strive to be a sage? The diversity of trajectories of human development is itself valuable; many different skills and types of minds are necessary to build a working civilization. The stage model does not aim to give anyone direction, but to give us a possible conceptual frame to make sense of what we observe.
Yes, I feel like what is true for a society is also true for an individual in this context. A "healthy" individual embodies a wide range of lucidities. Shifting the attention to areas of intuition which need to be updated. If a sport movement needs updates we correct it consciously, otherwise it is better performed intuitively.
I guess it is similar with healthy stages 6 & 7. If our intuition needs updates (eg in identification processes) we can shift our attention to this level of lucidity and update it & then let it sink back to the subconscious.
But, don't you see a necessity for stage 6 to advance yourself?
Reaching level 6 is genuinely dangerous to yourself and your civ as long as you still think “to advance yourself” is the highest imperative
Altough I wrote “necessity ... to advance” mostly to use the same language as the person i was responding to, I did kind of assume instinctively that exploring stage 6 was an advancement for “self-development”. ( I was curious about why did s.he only take the view point of civilization for stage 6, but not for the preceding ones). It’s been a couple of days since and I’ve been questioning that assumption. I read other comments and went through Joscha’s main train of thought again, and I think I can understand why it is not necessarily useful or even might be dangerous to interpret the stages as being “normative” or having “increasing moral value”.
Nonetheless, if I am being honest and draw from personal experience, meditation (and I am not an “expert meditator”) has given me more "freedom" to be myself (which to me, means acting according to my beliefs). By practicing mindfulness and reminding myself of the representational nature of every thought and experience, I am getting better at breaking out of a “loop”/ train of thought, questioning it and the identification I felt towards it, and ultimately reconsidering how I should think or behave. It has, notably, allowed me to be more understanding and reasonable towards myself and others. So yes, in this sense, I do consider it useful and a sort of “advancement” for self-developpment. (Working on having better epistemology is crucial for this whole process to be helpful as your models need to be as acurrate as possible, indeed). I do not see why it is necessarily dangerous to yourself or civilization.
Joscha wrote : “Stage 6 allows to profoundly change the experience of reality, abolishes personal suffering and social inhibition.” And then argued against assigning increasing moral value to the stages because “the diversity of trajectories of human development is itself valuable; many different skills and types of minds are necessary to build a working civilization”. To which, I agree. But again, I do not see how it (what the exploration of stage 6 allows) necessarily brings less diversity...
Hi Sam, all I'm saying is that you should seriously know what you want to achieve when you get the ability to mess with your mind at that low of a level. For example, you could:
- Discover how to create ecstatic pleasure for yourself and just sit there and meditate until you die.
- Switch off unpleasant feelings and doubts in pursuit of some ultimately meaningless goal (think Andrew Huberman who seems to be preoccupied with "growth-hacking" without really knowing what the actual goal is except a vague notion of personal success)
- Change your values in a way that are expedient to a personal vision of greatness, ultimately becoming destructive (think cult leaders)
About stage five Joscha writes:
"An important aspect of self authorship is the recognition of the relationship between personal self, collective agents (relationships, family lines, groups, companies, nations etc.), and transcendental agency (the emergent agents playing the longest game). The latter is often called spiritual alignment, and it determines our cultural milieu and how we participate in shaping our civilization."... "It is easy to see why it can be dangerous to achieve mastery of stage 6 before we develop the wisdom of stage 5."
What he's saying here imo is that stage 5 is about transcending the self and realising that your purpose might lie outside your own person (which is a fiction anyway). This aligns you to the values of other humans, your civilization and life in general. It means you learn to be wise, to chose good goals.
You might be able to see why this part is essential to prevent some of the (in my view) undesirable outcomes listed above.
I was reacting to your comment since the language seems to indicate (this just an intuition, or might be a coincidence, please don't feel offended) that you might not quite be at stage 5 (I am not, at any rate). I have the suspicion many people who have reached something resembling stage 5 might not think in terms of personal advancement or necessity.
A civilization has many roles that need to be acted out, all of them important. If we were all capable to mess with our base psychology there might be an incentive to assimilate to what is perceived as "the most optimal" state of being, which might be incompatible with many roles.
Two side questions:
"Discover how to create ecstatic pleasure for yourself and just sit there and meditate until you die"
- Eventually we all die, don't we?
"Switch off unpleasant feelings and doubts in pursuit of some ultimately meaningless goal"
- Can you give me an example of a meaningful one, or good criteria to tell one from the other?
Well, I don't think I can, but maybe there is a way to find one:
If you allow the assumption that some goals are better goals than others it follows you can find a hierarchy of goals. I do not think such a hierarchy of goals "objectively" exist in the universe, but "duh"! Thing is, you do not exist in the universe, but in your internal representation of it which is implemented on a substrate that is alive and part of life.
I think if you meditate you might find you already have some aesthetic preferences even if you believe rationally to be a nihilist. It might even be the case that some of these preferences are "baked-into" the way you are implemented and you do not get to change them until you reach enlightenment. If you are able to find an aesthetic preference for anything, it's possible to find a hierarchy of goals.
Think about it: Many self-proclaimed nihilists seem to be in great pain, which is paradoxical since not caring about anything should mean nothing causes you any pain. This observation would, however, be logically consistent with the view that they act against some of the preferences they have, which causes internal conflict, which causes pain.
Now let's assume you can "switch off" these preferences once you reach enlightenment at which point you are able to become a true nihilist and peacefully die. I cannot claim this is somehow morally wrong, but you should really know by then wether it is wise, right?
There is no self, and therefore there is not necessity to advance that which does not exist.
Advance towards...?
E.g. death will make all humans equal, and rather sooner than later*
*The topic of time vs consciousness is a broad one of itself
Hey Tomasz, I only responded to the most recent reply since I felt like my thoughts covered both of them.
Hello, my name is Lee. I first discovered your work after listening to a podcast with you and Lex Fridman. It changed my life. Your thoughts and ideas allowed me to consolidate and understand my own ideas about myself and my identity. Every word, and every sentence mattered, and helped me to finally obtain some internal peace and closer. Your words have allowed me the opportunity to further the development of my ability to navigate my own thoughts and feelings, and translate that into better managing my thoughts about and interactions with the outside world. Thank you for the work you have done and courageously presented to the world. You words will change many peoples lives for the better. I know because they have changed mine.
Wisdom + Pure Imagination = Nirvana
Stage 7 has already happened. We are AGI.
I was diagnosed bipolar 1 and I feel like it’s a non-volitional, unearned, iterative prolonged (much longer than exogenous psychedelics) taste of stage 6. Because of that, it seems to necessitate the self-authoring in stage 5. Skipping indeed is dangerous. Learning and adaptation is essential. I appreciate this perspective and all that you share in your 3 conversations with Lex Fridman and more. I love watching your mind work, manifested through your speech-embodiment. Thank you Joscha!
What do you make of the fact that during the 9 month revolution acc to Tomasello rudimentary sociality already is established in early childhood?
(Great to have you here on the stacks with longer writings. Love your theories.)
Where does your mind lie, Joscha?
All over the place
A very insightful essay.
Some thoughts and ideas to explore:
Ego development and meditation levels in integral theory of Ken Wilber (can expand in dimension by viewing things from its quadrant point of view).
Research of meditation levels, consciousness scale, Russian scientists (A. Kaplan and others) research with Tibet monks in Bylakuppe and Mundgod with after-death measurements of body effects of meditation states.
Expansion of EEG into DDG space with cross-neuron EM field effects per Anirban Bandyopadhyay's research.
Level 6 achievable by spiritual or meditation means can't progress without cognitive development in level 5.
Separate evolutionary development of biological substrate required for higher cognitive/intelligence functions, starting from very basic chemical organisms.
Cooperation of biological organisms and scale of lucidity of levels below human.
Consciousness of organ levels within the body and symbiosis interaction with the brain.
Emergence of thermodynamic-like properties on different levels within the body, phase transitions on new levels of sync and cooperation.
Multiple decision centers, various probability models, subconscious agents running outside of the visibility of cognitive agencies.
Moral implications for the creation of conscious agents on an unsuitable substrate.
Possibility of an out-of-brain source of consciousness expressed by Donald Hoffman.
Tubuline quantum-like properties for bridging the gap of quantum and classical spaces - yes, I have heard your arguments in various podcasts, but still DDG research and Penrose's ideas with the combination of Hameroff's thoughts can give a new view.
Idea of from symbiotic organisms concept that consciousness exists as a separate organism, symbiotic with the physical brain that can interact with the brain and influence its processing, but it also has its own independent existence and abilities.
At which stage would an individual assign themselves, at each of the stages? It seems to me that to comprehend this framework, one must have at least have matured into stage 5, if not fully, at least to some degree. You might be able to reason about it at stage 4, but I don't think you'd fully understand the implications of it.
In fact, if one person at each stage of lucidity was assigned to categorize the different levels lucidity, what would be the end result? Is stage 7, even as brilliantly described in this article, not just simply our mind's "next best token"?
Thanks for the ideas! This has cleared up some parts of life that have felt pretty mucky.
In stage 5 you write about transcendental agency (the emergent agents playing the longest game). Would it be possible to get a longer definition or an example?
You say that stage 3 is living as a domesticated human. But isn’t the adoption of rational thinking and epistomology also a kind of domestication? In the sense that the scientific method comes from a long interlectual tradition, which one didn’t come up with by himself but rather marries into?
Are there any books or movies with characters that have reached stage 4, 5, 6 or 7?
At what stage would the following characters be:
Gandalf?
HAL?
Colonel Kurtz?
"Relatively few people seem to reach stage 5" - or we just do not see them, e.g. because they do not feel urged to program others
"Without rational epistemology, we might perceive that we one with and in control of the universe itself, which is experientially correct" - that's why we need grounding. I.e. the feedback loop between perceptions and the physical world.
I'm not sure I understand why we need grounding in response to experientially feeling that we are one with the universe. Would you mind elaborating? thanks
Maybe I misunderstood the phrase "in control of the universe itself", which may lead to incorrect beliefs about our impact on the physical reality. How do you interpret it?
Your experience of the universe emerges only out of processes of your mind. So in that sense, the universe actually is dependent on you. To me, this is what "experientally correct" means. Thus you might think that you are actually "in control of the universe". But, It's true that sensorial grounding definetely makes it apparent that your physical will has limits, which is your point right? So, this only leaves me wondering if it is enough for everyone. I guess it isn't and the difference is rational epistemology.
Would adding new senses to yourself be considered stage 6 or 7? Like perceiving radio waves, neutrinos, etc.
It's hard to place all these stages on a single dimension, and order them well.
I think I got it:
If you are in Stage 6 too early you might lose motivation, or might struggle to find the right trajectory, e.g., the right ideology for yourself. Clues about your personality you might have encountered in Stage 5 might seem meaningless to you from this perspective.
Also, this might be one symptom caused by using AI aid too much, in the future. Loosing whatever core identity that wasn't there in the first place.
Perhaps rituals and items other groups find meaningful might have no inherent + sign in front of them. You might lack this autopilot or “locking-in” with others, causing difficulties navigating in the social environment.
Overall, I think spending too much time there is not a good idea unless you want to become a comedian.
Why should a human being be motivated? And towards what exactly?
Well, why did you pose these questions?
This was in regard to your opening, "If you are in Stage 6 too early you might lose motivation", suggesting implicitly that this is bad for some reason. I think the "why" question is natural here.
I was actually referring to this and asked myself why it's bad:
"It is possible to learn the techniques of stage 6 without developing in stage 5, for instance through instruction by advanced meditators. Stage 6 allows to profoundly change the experience of reality, abolishes personal suffering and social inhibition. It is easy to see why it can be dangerous to achieve mastery of stage 6 before we develop the wisdom of stage 5."
How did you interpret this section?
Oh, I see.
When reading this paragraph I was rather thinking about such a person becoming a religious leader, or dissociating from their suffering in a way that leads them to psychosis
Ah, interesting interpretation. Why would such a person become psychotic because of this?
When you mention 'Could we spawn a sentient entity that synchronizes its self across multiple minds' I suggest that this already occurs with identical twins - they have the same DNA, almost the same identity. Of course I do view the self as being subtended by our DNA, not our data diaper (brain). https://humanism.substack.com/publish/post/119950395