Their Terms Say They Don’t Use Your Data. An Investor Told Me Otherwise.
Do companies use your likeness hoping you will never find out?
When I was talking to investors about Destined AI, I had a conversation that changed how I think about privacy.
One investor in particular told me that I should build a product to collect people’s voice data for free and then train my own models with it. He said that would be my moat. That’s the word he used. Moat. Your competitive advantage. And his version of a competitive advantage was getting people’s voice data without them fully understanding what it would be used for.
He used a voice journaling app as the example. An app where people speak into it to journal — talk about their day, their feelings, their thoughts. Personal stuff. Intimate stuff. And according to this investor, that company took the user data and built their voice AI models from it.
But here’s the thing. If you go and look at that app in the App Store or read their terms, it says they do not use your data.
Let that sit for a second.
Self-Reported Means Self-Policed
Most people don’t know this, but the privacy labels you see on apps in the Apple App Store are self-reported. The developer fills out a form telling Apple what data they collect and how they use it. Apple publishes it. That’s it.
Apple is not going through the company’s code to verify it. Apple is not auditing their servers. Apple is not checking whether the data that flows through the app matches what the label says. The label is what the company told Apple. And Apple took them at their word. It’s the honor policy.
So when you see that green checkmark or that “no data collected” badge and you feel safe — understand that you’re trusting the company’s honesty. Not Apple’s verification. There is no verification.
This is the same principle with terms of service across the board. When a company writes “we do not sell your data” or “we do not use your data for training,” that’s a claim. It’s not a fact. And there is very little infrastructure in place to hold them to it. Especially in the US where data privacy laws are still a patchwork compared to other places where you have more rights.
Voice, Video, and Image Data Are the New Gold
That conversation made me think about what kinds of data are most valuable right now in the AI race. And the answer is obvious when you look at where the models are going. Text data has been scraped from the entire internet. Everyone has it. The next frontier is voice, video, and image data. That’s where the moats are being built.
Think about the apps on your phone right now that have access to your voice. Voice assistants. Voice journaling apps. Language learning apps. Voice memos. Telehealth platforms. Meeting transcription tools. Any app that listens to you has the raw material to build or improve a voice AI model.
Now think about the ones that have your images. Photo editing apps. Social media. Cloud storage. Stock photo platforms. Family album apps. They all have your visual data. And we’ve already seen what can happen with that — my 11,855 images trained Adobe’s AI. The terms didn’t say that’s what the license was for. But they did it anyway.
Video is the same story. Any platform where you upload video content has footage that could be used to train models for everything from facial recognition to motion capture to video generation AI.
This isn’t speculation. This is the business model that investor was recommending to me. Collect the data through a “useful” product. Train models on it. That’s the moat.
I Saw This Firsthand
I’m not guessing about whether companies honor their terms. I lived it.
Adobe’s agreement with me was for distributing my images to end users and promoting my work. The sections of the contract were literally labeled “License We Need to Distribute Your Work to Our End Users” and “License We Need to Promote Your Work.” That’s the context. That’s the purpose.
And then they used that same license to train Firefly and Sensei. They argued the phrase “new features and services” covered anything in the universe. On their own AI Ethics page, they talked about respecting creators’ “choice and control.” Meanwhile, behind the scenes, the content was already in the training pipeline.
Terms that said one thing. A company that did another.
So when someone tells me an app’s privacy label says “no data collected,” I hear it. I just don’t automatically believe it. Not because every company is lying. But because I’ve seen what happens when they are, and there’s almost nothing in place to catch them.
How I Think About It Now
I’m not saying delete every app off your phone. I’m saying be reasonably cautious. Especially with voice, video, and image data. That’s the data companies are building their futures on right now. And they’re getting it from you, often through products that feel harmless.
Here’s what I do. Before I give an app access to my microphone, my camera, or my photo library, I ask myself a few questions. What does this company actually do with this data? Is there a business model that depends on my data being the product? Do their terms match their marketing? And is anyone holding them accountable if they don’t?
Usually the answer to that last question is no. Nobody is holding them accountable. The App Store label is self-reported. The terms of service are written by their lawyers. The privacy policy is designed to protect them, not you. And enforcement in the US is basically nonexistent unless you can afford to take them to arbitration or court — which most people can’t.
That doesn’t mean every company is lying. There are good ones. And when I find them, I support them. That’s the “buy” in my buy, build, or bypass framework.
But for the rest? I’m cautious. I give the minimum data necessary. I use the privacy controls available. And for things that really matter — my family’s photos, my creative work, my voice — I think carefully about who gets access and whether I trust them with something I can never fully take back.
What I’d Tell You
Next time you download an app and it says “no data collected” in the App Store — know that’s what the company told Apple, not what Apple confirmed. It might be true. It might not be. There’s no way for you to verify it from the outside.
If the app asks for access to your microphone, your photos, or your camera — think about what they could build with that data. Think about the investor who told me to collect voice data for free and turn it into a moat. That’s the thinking that drives this industry. Your data isn’t a byproduct of the product. In many cases, your data is the product.
And if a company’s terms and their behavior ever contradict each other — trust the behavior. Every time.
I learned that lesson with Adobe. I’m sharing it so you don’t have to learn it the same way.
“If a company’s terms and their behavior ever contradict each other — trust the behavior.”