Longtime human-AI interaction expert Julie Carpenter is concerned about the potential privacy implications of OpenAI's upcoming 'adult mode' for its chatbot, ChatGPT. The feature could enable users to generate explicit content, but Carpenter warns that it may also lead to intimate surveillance.

The risk, according to Carpenter and other experts, lies in how detailed and persistent AI-generated memories can be. User preferences, including sexual proclivities, could be logged and used in future interactions, potentially without the user's full knowledge or consent.

While OpenAI has implemented measures such as temporary chats that are not saved in a user’s log history, these conversations may still be retained for up to 30 days. Additionally, the company faces legal uncertainty regarding data retention.

The potential consequences of such exposure include security breaches or government demands for access to chat logs, which could reveal highly sensitive and personal information.

OpenAI's plans come as concerns grow within its advisory council about the risks involved. The release of explicit content on a mainstream platform like ChatGPT could set a precedent that other AI companies might follow, potentially leading to the normalization of intimate surveillance through technology.