Recently, Seedance 2.0 wasn’t the only AI tool Bytedance developed. They released an AI voice-recording hardware device, Soundcore Work, that is deeply embedded in their meeting software and can generate transcripts and AI summaries for all recordings. It is yet another product similar to what's already on the market, but it’s worth noting that ByteDance has managed to make the recorder even smaller, down to buckle size. As software engineers who worked on similar systems, we can see a clear trend: these agentic AI products are maturing. This trend moves towards a sort of agentic servant that can actively listen and even take action based on people’s behavior. Especially in this case, we can see that these products are passing the adoption lifecycle and diffusing from nerds into the general public. This makes me think about the world if these AI recorders or agentic products were widely owned like mobile phones, and how privacy laws would be challenged under new norms.
We can see major differences between the AI recorder product and previous recording devices, such as phones and smart home devices, using Solove's Taxonomy framework. In information collection, recorded speech is uploaded to cloud services for processing rather than being kept locally. In information processing, audio is transcribed, embedded, and indexed using RAG systems, enabling searchability and long-term retrieval compared to simply storing audio files. Finally, in information dissemination, the structured transcript is easily accessible and reusable across contexts, greatly increasing the fluidity and value of these records. To put it simply, an AI recorder will expose and store much more audio data to the cloud than before.
From the data subject perspective, we can envision changes in social norms, especially on what is appropriate information and how it is distributed. For example, it’s common to share video recordings, such as lectures or meetings, but with an AI recorder, the employer can record all audio during working hours, which, in previous norms, was not a private space. Most importantly, AI makes information searchable and long-lived. I can easily refer back to what people were gossiping about weeks ago if I left my recorder on. Will this change people’s understanding of privacy in public spaces? Our boss can look up the assigned deadline we promised at the start of the quarter. Can this reference be legally trustworthy? Is this a hallucination from the AI that missummarized our conversation with the boss? As an ordinary user, the information processing seems even more black-boxed than before and touches more of our lives.
Meanwhile, the data holders are also growing differently, as they can access large amounts of data from only a few customers. For example, even a small percentage of people with an AI recorder can capture detailed conversations happening across the entire office. This means companies selling only thousands of devices can have petabytes of recordings in this area, and they may avoid the CPRA, since their primary revenue comes from hardware and APIs. It would be valuable to obtain detailed historical conversations for your everyday life, and possibly open up new markets and usage. Aside from using these data for predictive targeting or ads, we can develop things such as AI companionship that record and interact with your life. Similar to people dating with ChatGPT, but instead of passively waiting for user input, these new wearable devices can actively gather information to better understand people. People can consent to what is collected when purchasing, and companies can argue that the minimal amount of data needed to personalize services is to record as much as possible.
The feeling of discord stems from AI blurring the boundary. Previously, data were collected to track user behavior or simply to represent users in the database, which required labor-intensive work for engineers. Now, with AI’s superpower in digesting information and processing everything, we can store as much information as we can gather and chunk it for AI to summarize. Take ChatGPT, for example: people ask it everything, and it automatically summarizes their career, their family members, and what they're most interested in. This obscured the boundary for what data is collected and how it is managed. If you ask it about medical information and it remembers your entire medical history, should HIPAA apply? If you delegate a grocery reminder or help you file a tax report, should GLBA apply? What about data breaches? What if the government wants to monitor? Perhaps the CPRA will expand to address more of these emerging issues or create new patchworks for the special AI categories. We can definitely see more of these privacy and ethical issues arising as AI is gradually embedded in our lives and gathering even more data, starting with the recording of our working hours.