Why AI Needs New Rules for Private Conversations: ChatGPT and Confidentiality
As Sam Altman warns users that ChatGPT offers no legal privacy protection, the AI community must rethink trust, data and the social role of digital tools.
When OpenAI CEO Sam Altman publicly admits that conversations with ChatGPT have no legal confidentiality, it’s a wake-up call for anyone engaging with AI, professionally or personally. In a world where digital tools increasingly support our work, learning and even emotional wellbeing, the question of privacy isn’t theoretical. It’s immediate and real.
The Illusion of Digital Privacy
The rise of generative AI has been meteoric. For millions, ChatGPT is now a daily companion, sometimes for trivial queries, but sometimes for deeply personal matters: career crises, relationships even mental health struggles. The recent TechCrunch article highlights a growing trend: many users see AI as a judgement-free, always-available confidant.
However, as Altman makes clear, these interactions are nothing like the legally protected, confidential conversations you have with a doctor, therapist or lawyer. In fact, under current law, AI providers like OpenAI may be compelled to share user conversations in response to a court order, even chats that were “deleted” by the user. With new lawsuits demanding that OpenAI retain user data indefinitely, the legal landscape is only getting murkier.
Why This Matters for Users and for AI’s Future
This lack of “AI privilege” is more than a technicality. It has profound implications for:
• Trust: If users can’t rely on privacy, they may hold back critical details or avoid using these tools for anything sensitive. This undermines the usefulness and adoption of AI for support, learning and wellbeing.
• Safety and ethics: Sensitive information - personal histories, medical or legal concerns - could be exposed not just to platform providers, but potentially to courts, adversaries or malicious actors.
• Professionalism: For the AI field, public trust is the foundation of progress. A lack of clarity or responsibility around data use is not just a legal risk; it’s an existential one.
Is “AI Privilege” the Solution?
Altman’s call for legal “AI privilege”, akin to doctor-patient or attorney-client confidentiality, raises intriguing questions. Should some AI conversations be granted legal protection? What types of data, and under what circumstances? And, what technical and policy changes would be needed to make this a reality?
Creating this kind of privilege won’t be simple. It will require collaboration between technologists, policymakers, lawyers and ethicists. Data storage, access and deletion policies will need to be transparent, independently audited and clearly communicated to users.
What Responsible Practice Looks Like Now
While the law evolves, there are steps the AI community and professional bodies can take:
1. Transparency: AI platforms should be clear about what happens to user data, who can access it and for how long it is retained.
2. User controls: Give users real, meaningful ways to manage, export or permanently delete their data.
3. Professional guidance: Educators, healthcare providers and legal professionals should advise clients and students about the limits of AI privacy.
4. Advocacy: The AI community must engage with lawmakers to shape future privacy protections, balancing innovation with public trust.
Building Trust by Raising Standards
As AI becomes ever more woven into our daily lives, the standards for trust and privacy must rise just as fast as the technology. The promise of AI, whether as a tool, a tutor or even a kind of digital confidant, depends on users feeling secure that what they share will not be exposed without their consent.
At the Institute of Artificial Intelligence, we believe in building this trust through accreditation, education and open dialogue about rights, responsibilities and standards.
Now is the time for the AI profession to lead the conversation not just on what AI can do, but on how it should be used, and what users can safely expect. Because in the end, trust is the most powerful technology of all.
Interested in learning more about responsible AI standards and accreditation? Join IoAI and be part of shaping the future of trustworthy, professional AI.
