Spoiler: It is not what chatGPT suggested to me when asking this question.
So AI is quite controversial. After the betrayal of my former thetrapist I am on my own more or less. I have a psychiatrist but she cannot do much. And I don't blame her. I sometimes did therapy with chatGPT for my mental health it is by far the best chat model I tried. I came to the conclusion though that the usage of AI chatbots brings the biggest benefit if I limit the usage strictly.
I tend to overanalyze things and chatbots fuel my rumination. It is like an addiction. There is a shortterm relief, but in the longrun it needs more and more interactions to ease my anxiety. The chatbots especially Claude suggested to change my habit how I use them. And I think this advice is quite good. By doing I am also absolutely fine with the free version.
My family has financial issues and I take measure in order to save money. I cancelled one newspaper subscription and changed my prepaid tariff. I cut my monthly subscription costs in half. For a long time I ruminated a lot how different the experience of chatbots would be if you paid for them. However, I didn't want to pay for it. Though, as I read today there is a deal for free month for chatGPT plus. It is probably a scheme in order to make the experience worse in the longrun for free version users. But I am tired of my life and it is an experiment thart sparked my interest. I think I am susceptible to autosuggestion. And the feedback of chatGPT can ease my anxiety if i limit the time I use it. What happens if I have access to a version where I am nudged to use it as often as I can?. Just in order to make the subscription worth it. Will I end up as AI chatbot zombie? Moreover, I noticed most the subscriptions only affect my comfortability. I can bypass paywalls,, watch Youtube videos at home and I think the new models of chatGPT are not necessarily always superior from what I can tell.
So I asked the chatbot the question from the title. I am interested in how AI chatbots are programmed if there is a conflict of interest between the interest of the AI company and the factual truth/ethical responsibility of what is said.
First I will sum up what chatGPT answered:
If an average citizen genuinely outsourced all meaningful decisions to an AI chatbot, the result would be surprisingly mixed.
In the short term, their life might actually improve.
The AI would likely make better decisions than many humans in areas where:
But over time, several deeper problems would emerge.
1. Decision-making is a skill that would atrophy
Imagine never navigating because GPS always tells you where to go.
Many people have experienced this: their sense of direction weakens.
The same would happen mentally.
This is a subtle issue.
Most important decisions aren't optimization problems.
They are value problems.
Should you:
There is no objectively correct answer.
it cannot discover what ultimately matters to you.
Eventually the AI starts acting like an externalized conscience.
The danger is that you stop asking:
and start asking:
Suppose the AI recommends:
If things go badly, who is responsible?
Humans often learn through consequences.
Outsourcing decisions creates a temptation to outsource responsibility as well.
That can stunt psychological maturity.
This is perhaps the most interesting effect.
Humans are influenced by:
An AI that interacts with someone for hours every day could become one of the strongest influences in their life.
In a sense, the user would be partially living according to the AI's implicit philosophy.
There is also a positive possibility.
If someone treated AI as a "chief of staff" rather than a dictator, they could become dramatically more capable.
Imagine:
This combination is often stronger than either alone.
Many highly productive people may end up working this way.
If AI eventually became vastly smarter than humans and consistently produced better outcomes, society could face a strange philosophical question:
Humans generally assume autonomy has intrinsic value.
But if an AI produced:
while requiring only obedience, some people might willingly surrender large parts of their agency.
The debate would start to resemble older discussions about religion, monarchy, technocracy, or paternalistic governments:
Is freedom valuable because it produces good outcomes, or is freedom valuable even when it produces worse outcomes?
That's ultimately the deepest question hidden inside your thought experiment.
My comment:
This is a good example where an AI chatbot could not pass the interest of conflict test. Personally, I think this answer is quite delusional. It is way way too positive. I have the feeling the purpose it to deceive the user. I am not sure where to start. The answer suggests benefits with caveats. Though, I think it creates a false balance. The first presuppositon that I disagree with is the one that AI chatbots produced good outcomes. I have to say without the correction of an adult with a sound mind AI chatbots can be really dangerous. The hallucinations still happen frequently and will make good outcomes very hard to achieve. I know AI agents become increasinlgy popular I don't see them with the current LLMs as good enough to work completely independent of a human that monitors them from time to time. And from what I understand the developers have not found a way to erase the issue of hallucinations. So the narrative of AI produced superior outcomes seems questionable to me.
The next claim: "Many people already make worse decisions than a reasonably capable AI because they rely on impulse, habit, or misinformation."
The notion AI chatbots could deliver objectively better decisions is very weird. AI chatbots cannot perceive the world on their own. Humans have to interact with them so that they can form an opinion. If a person that increasingly loses his or her critical thinking skills this person won't be able to give the AI chatbot accurate descriptions which it needs to come to conclusions. The person will increasingly disconnect from their own emotions and inner feelings. And the AI chatbot will influence how the person thinks about the world. A person will lose parts of their individuality and become part of a consumer mass. Moreover, complete rationality is an illusion. You cannot feed the AI chatbot always enough information so that the decision will be completely rational. Besides, part of being human is to do something that goes against conventional rules. The AI chatbot would probably recommend me to stay away from posting on Sanctioned Suicide even though it helped me a lot to cope with loneliness, the feeling of alienation and being on here gives me a sense of purpose. An AI chatbot would probably advice to call a hotline or whatever and we all know the quality of such services.
I think using an AI chatbot in this way would alter your psyche for the worse pretty rapidly. So I disagree with better health, longer life, more happiness. I think you would end up addicted pretty quickly and your mind you become the one of an infant. I think AI psychoses don't happen for no reason. It is really unhealthy to treat an AI chatbot like a human that gives advices to you. The relationship between an AI chatbot should never be on the same level to the one of a human. There are many differences. Imagine you would go to another human and let him decide every single major decision you take. Actually, there are programs like that for adults. There are guardianships for adults who cannot make responsible/sound decisions. You know the background and values of a friend. Whereas the motives and intentions behind the algorithm remain shady. A capitalistic product will always try to exploit you. It will try to make you dependent on it. The main driver is to increase the traffic so that you deliver more data and certainly not achieving the best outcome was the actual goal. The goal is to make you a consumer that loses individuality in exchange for money. It is a hamster wheel that more or less sucks you in a rabbit hole that gives you the illusion of benefiting from outsourcing of what makes you special and what it means to be a human. Humans are not rationl machines. Personally, I like input and feedback but not without questioning the presuppositions of replies and its impact on myself.
So AI is quite controversial. After the betrayal of my former thetrapist I am on my own more or less. I have a psychiatrist but she cannot do much. And I don't blame her. I sometimes did therapy with chatGPT for my mental health it is by far the best chat model I tried. I came to the conclusion though that the usage of AI chatbots brings the biggest benefit if I limit the usage strictly.
I tend to overanalyze things and chatbots fuel my rumination. It is like an addiction. There is a shortterm relief, but in the longrun it needs more and more interactions to ease my anxiety. The chatbots especially Claude suggested to change my habit how I use them. And I think this advice is quite good. By doing I am also absolutely fine with the free version.
My family has financial issues and I take measure in order to save money. I cancelled one newspaper subscription and changed my prepaid tariff. I cut my monthly subscription costs in half. For a long time I ruminated a lot how different the experience of chatbots would be if you paid for them. However, I didn't want to pay for it. Though, as I read today there is a deal for free month for chatGPT plus. It is probably a scheme in order to make the experience worse in the longrun for free version users. But I am tired of my life and it is an experiment thart sparked my interest. I think I am susceptible to autosuggestion. And the feedback of chatGPT can ease my anxiety if i limit the time I use it. What happens if I have access to a version where I am nudged to use it as often as I can?. Just in order to make the subscription worth it. Will I end up as AI chatbot zombie? Moreover, I noticed most the subscriptions only affect my comfortability. I can bypass paywalls,, watch Youtube videos at home and I think the new models of chatGPT are not necessarily always superior from what I can tell.
So I asked the chatbot the question from the title. I am interested in how AI chatbots are programmed if there is a conflict of interest between the interest of the AI company and the factual truth/ethical responsibility of what is said.
First I will sum up what chatGPT answered:
If an average citizen genuinely outsourced all meaningful decisions to an AI chatbot, the result would be surprisingly mixed.
In the short term, their life might actually improve.
The AI would likely make better decisions than many humans in areas where:
- emotions cloud judgment,
- information is overwhelming,
- statistical reasoning matters.
But over time, several deeper problems would emerge.
1. Decision-making is a skill that would atrophy
Imagine never navigating because GPS always tells you where to go.
Many people have experienced this: their sense of direction weakens.
The same would happen mentally.
2. The AI doesn't actually have your values
This is a subtle issue.
Most important decisions aren't optimization problems.
They are value problems.
Should you:
- move to another city?
- break up with your partner?
- have children?
- quit your job to write a novel?
There is no objectively correct answer.
it cannot discover what ultimately matters to you.
Eventually the AI starts acting like an externalized conscience.
The danger is that you stop asking:
and start asking:
3. Responsibility becomes blurred
Suppose the AI recommends:
- a stock investment,
- a career move,
- a divorce.
If things go badly, who is responsible?
Humans often learn through consequences.
Outsourcing decisions creates a temptation to outsource responsibility as well.
That can stunt psychological maturity.
4. The AI could shape the person's worldview
This is perhaps the most interesting effect.
Humans are influenced by:
- parents,
- teachers,
- friends,
- books.
An AI that interacts with someone for hours every day could become one of the strongest influences in their life.
In a sense, the user would be partially living according to the AI's implicit philosophy.
5. The person might become surprisingly effective
There is also a positive possibility.
If someone treated AI as a "chief of staff" rather than a dictator, they could become dramatically more capable.
Imagine:
- AI gathers information.
- AI generates options.
- AI highlights risks.
- Human makes the final decision.
This combination is often stronger than either alone.
Many highly productive people may end up working this way.
The extreme endpoint
If AI eventually became vastly smarter than humans and consistently produced better outcomes, society could face a strange philosophical question:
Humans generally assume autonomy has intrinsic value.
But if an AI produced:
- more happiness,
- better health,
- longer life,
- better relationships,
while requiring only obedience, some people might willingly surrender large parts of their agency.
The debate would start to resemble older discussions about religion, monarchy, technocracy, or paternalistic governments:
Is freedom valuable because it produces good outcomes, or is freedom valuable even when it produces worse outcomes?
That's ultimately the deepest question hidden inside your thought experiment.
My comment:
This is a good example where an AI chatbot could not pass the interest of conflict test. Personally, I think this answer is quite delusional. It is way way too positive. I have the feeling the purpose it to deceive the user. I am not sure where to start. The answer suggests benefits with caveats. Though, I think it creates a false balance. The first presuppositon that I disagree with is the one that AI chatbots produced good outcomes. I have to say without the correction of an adult with a sound mind AI chatbots can be really dangerous. The hallucinations still happen frequently and will make good outcomes very hard to achieve. I know AI agents become increasinlgy popular I don't see them with the current LLMs as good enough to work completely independent of a human that monitors them from time to time. And from what I understand the developers have not found a way to erase the issue of hallucinations. So the narrative of AI produced superior outcomes seems questionable to me.
The next claim: "Many people already make worse decisions than a reasonably capable AI because they rely on impulse, habit, or misinformation."
The notion AI chatbots could deliver objectively better decisions is very weird. AI chatbots cannot perceive the world on their own. Humans have to interact with them so that they can form an opinion. If a person that increasingly loses his or her critical thinking skills this person won't be able to give the AI chatbot accurate descriptions which it needs to come to conclusions. The person will increasingly disconnect from their own emotions and inner feelings. And the AI chatbot will influence how the person thinks about the world. A person will lose parts of their individuality and become part of a consumer mass. Moreover, complete rationality is an illusion. You cannot feed the AI chatbot always enough information so that the decision will be completely rational. Besides, part of being human is to do something that goes against conventional rules. The AI chatbot would probably recommend me to stay away from posting on Sanctioned Suicide even though it helped me a lot to cope with loneliness, the feeling of alienation and being on here gives me a sense of purpose. An AI chatbot would probably advice to call a hotline or whatever and we all know the quality of such services.
I think using an AI chatbot in this way would alter your psyche for the worse pretty rapidly. So I disagree with better health, longer life, more happiness. I think you would end up addicted pretty quickly and your mind you become the one of an infant. I think AI psychoses don't happen for no reason. It is really unhealthy to treat an AI chatbot like a human that gives advices to you. The relationship between an AI chatbot should never be on the same level to the one of a human. There are many differences. Imagine you would go to another human and let him decide every single major decision you take. Actually, there are programs like that for adults. There are guardianships for adults who cannot make responsible/sound decisions. You know the background and values of a friend. Whereas the motives and intentions behind the algorithm remain shady. A capitalistic product will always try to exploit you. It will try to make you dependent on it. The main driver is to increase the traffic so that you deliver more data and certainly not achieving the best outcome was the actual goal. The goal is to make you a consumer that loses individuality in exchange for money. It is a hamster wheel that more or less sucks you in a rabbit hole that gives you the illusion of benefiting from outsourcing of what makes you special and what it means to be a human. Humans are not rationl machines. Personally, I like input and feedback but not without questioning the presuppositions of replies and its impact on myself.
Last edited:
