Dupe: https://news.ycombinator.com/item?id=32538805

Another version of this same story, also marked as a dupe: https://news.ycombinator.com/item?id=32557294

Cyberpunk dystopia realized. It is not so hard to imagine a near future where corporations have much more power than states.

Imagine you do something like this in the near future. You are banned from all communications. Alpha corp. reports it to VISA corp. and all your money is confiscated. State can't help you as you do not have a way to communicate with it, don't have a way to start any legal process as you do not have a way to pay. All your daily subscriptions to housing, mobility, entertainment are cancelled and you are homeless the same day.

The only way for you to stay alive is to hike to lawless desert city of "Kowloon 2.1". You walk by the highways observing huge OLED billboards. They show presidential elections. This time it is some adult TV reality show star competing with tanned bodybuilder.

I can't believe the response after the media approaching them was still "We're right and your account remains banned" I understand the reluctance to expose themselves to potential legal complications but in this case with the police report it's pretty clear cut that these photos were not criminal and the photo taker has not committed the crime Google is accusing him of.

further discussion on the original NYT story here: https://news.ycombinator.com/item?id=32538805

I was reminded of David Graeber's description of banking bureaucracies, as I read this.

At ground level, bank employees blame irrational form filling, small and nonsensical process requirements on "regulation." In reality, they are implementing a bank "compliance" policy that is relates to regulation. That way, neither the regulator nor the bank is culpable... either for solving the problem or for consequences of the solution. Meanwhile, the banks themselves probably lobbied for the regulation specifics, and almost certainly wrote large parts of it themselves. It's cozy.

At Google-scale, there is no dividing line between between private & government bureaucracies. They're merged in a way that evolves to deflect criticism & responsibility from both.

The scariest line, to me, was this:

"These companies have access to a tremendously invasive amount of data about people’s lives. And still they don’t have the context of what people’s lives actually are."

Well... "luckily" Google & such are getting better all the time at "context." The response to such issues is usually to deepen the bureaucracies, not to pull back.

> “We follow US law in defining what constitutes CSAM and use a combination of hash matching technology and artificial intelligence to identify it and remove it from our platforms,” said Christa Muldoon, a Google spokesperson.

> Muldoon added that Google staffers who review CSAM were trained by medical experts to look for rashes or other issues. They themselves, however, were not medical experts and medical experts were not consulted when reviewing each case, she said.

Which is a nice way of saying: "AI flagged the pictures, we can't really judge if they are medical pictures or not. Enjoy your permaban!"

Besides, permabanning someone because of a single flagged picture seems like a dumb policy anyway. Even if there was no rash or inflammation in the picture, the child could have had a pain in the groin, so wanting to visually examine it could make sense for a doctor.

To how many parents has this already happened? I get the feeling "Mark" who lives in SF probably knows folks in media or tech. How many nameless, powerless people in Middle America or the Global South had their digital lives nuked and suffered in silence?

As a parent, I feel for this fellow parent.

About 10 days ago, my 3 year old son developed Bell's palsy in a period of 24 hours. This means the right side of his face became paralyzed, for no good/known reason. Most cases of Bell's palsy, esp. with toddlers, go away within about a month.

https://en.wikipedia.org/wiki/Bell%27s_palsy

I'm taking videos of his face every day to track progress, to see whether it's recovering. Also, I'm sharing some of the videos on a Messenger chat with other concerned family members. Obviously, since this is his face, I'm not in danger of violating any TOS on this. I have Google Photos auto-upload turned on.

But, knowing how stressed I was when this started, I was definitely not thinking about Google's (or Facebook's) TOS when I was making the videos, so if it had been some other condition where the affected area was the groin, like in the article, I'm pretty sure I would not have realized I'm risking my entire online presence.

Some good news to close: my son's face is showing good movement after ~10 days, so I think he'll recover within a month.

"Mark was cleared of any criminal wrongdoing, but Google has said it will stand by its decision."

Can someone explain what Google's logic is? At this point [after the news articles], clearly humans are involved at Google, clearly they know this is not child porn, so why not reinstate the account and move on?

Plenty of facets on this one, and I fear they all require changing and enforcing legislation and to fix them.

* Google provides a good product free of charge to everybody, but claims it can withdraw the offer to anyone with no notice and without giving any reason. The law should require more rights, guarantees and due process even for free service providers. At minimum, the right to recover my data, the right to have some time to fix the problem, the right to have everyone following the same rules, knowing the rules and knowing the exact violation, and some basic warrantees. As a parallel: Malls are open for everyone just like Google, and it still doesn't allow them to ban people at random. The USA learned with black people how quickly this becomes ugly. But a mall still can remove customers too troubling, the cops can be called, and both mall and customer can sue if it is really worth it.

* People should pay (and be able to pay) for a product that is worth it. The services provided by Google are easily worth money.

* It's hard to ban anyone on the internet without a stable identity, government provided or not. People just take a new identity and come back. This is especially sensitive in the USA, but it causes all kinds of trouble. Maybe coupling the ID to a payment is an alternative, so you can get a new ID if you pay for it. €10 /id / year would probably be enough to shut out almost every troublemaker, and the law can deal with what's left.

* The child porn laws are insane and should be changed. It is not normal that you can send a picture to someone, and the law requires the receiver should go to jail even without him knowing anything happened. Under such laws, the google 'scorched earth' policy is the only option they have.

* EULAs need to cease existing. If I buy an Andriod phone, I give money to the manufacturer, the manufacturer gives money to Google. That means I should have a right to access google play etc without all kinds of EULAs making the device I bought non-functional on any whim from Google. Just paying should be enough. It used to be that license agreements existed only between companies, and were deemed too complicated for ordinary consumers. The 'End User' aspect has to die off. Normal copyright will still protect the software just as it does with books and DVDs, so there really is no need.

The times, they are a-changin'

Not long ago, every other comment about a case like this would say "well, Google is a private corporation, they can do as they please".

Now, those comments have disappeared and most comments call for more regulation.

The same thing happened with advertising. In the 2010s many people here advocated in favor of advertising, and often said ad-blockers were bad, useless, or criminal. And then the dominant view became that the Internet is absolutely unusable without a proper adblocker.

Not sure if any positive change will come out of this (advertising certainly didn't go away) but it's fascinating to see.

I've grown up to believe that we deserve this, at a societal level. As long as we've let technology invade each and every inch of our private space this was bound to happen, sooner or later.

Props though to the SF police which has stopped the investigation when, I guess automatically, they had been made aware of the whole thing by Google. I suspect in a few years' time that won't be the case anymore, even the Police investigation in this sort of cases will have become "automated".

With Google representing all branches of a shadow trias politica is like a secret police in totalitarian countries. If thanks to this news on Google's move only one person decides _not_ to send pictures on their doctors request, then this action has sent out the same message as the secret police making your neighbours disappear after saying something unfriendly about the dictator.

I've been a happy Google photos user for a few years but the fact that they have changed their business model from "free forever" to "now you pay", and the fact that photos take half of my Google free tier, and this scary stuff about CSAM I think the time has come for self-hosting everything.

I will wait a bit for email (too complicated), but as for pictures, any software you would recommend? Ideally, the server should run on a Raspberry Pi and has face recognition and automatic tagging.

Another reason why cloud services must come with end-to-end encryption ON by default. What business does GOOG or any other big tech have with reading their customers' emails or analyzing their photos/videos etc., except for creating profiles with which they can push ads/sell stuff?

Consumers must demand this (i.e., e2e encryption and no peeking) and shun products that don't offer it.

I hope the father sues GOOG and wins hundreds of millions in damages for mental anguish, defamation, breach of privacy, denial of service, discrimination (do you think GOOG has never reinstated an account previously?) and every other legal grounds they have violated.

Hey, remember when Apple was going to implement photo matching against known CSAM and everyone freaked out, prompting them to reverse the decision?

Well, this is Google having an AI classify your images as potential CSAM and having had that for a while and everyone’s cool with it.

To stay on topic, I'm a parent and have autoupload to google photos.

How can I prevent the photo from bring uploaded? I could disable autoupload, but as soon as I resume it, the photo will be uploaded, unless I remove it from the phone.

Any suggestions?

The day Google banned(from Gmail and all Google products without ability to port thier data) some pixel phone buyers for reselling their phones[1], I did these 3 things:

1. Setup a new personal domain and email on fastmail. I can take my email somewhere else if fastmail decides to ban me.

2. Setup account on Backblaze for my backups, Google drive can't be trusted.

3. Moved to alternatives for all Google services. Removed Google signin from all the other websites.

[1] https://www.theguardian.com/technology/2016/nov/17/google-su...

It’s quick and easy to move from Gmail to Fastmail. Did it earlier this year. Now YouTube is the only Google product I use, and it works pretty well without an account.

Two unintended consequences

- Patients find it too risky to store medical information on Google drives/devices.

- Exhausted patients are forced to scrutinise every upload for the potential for Google to mistakenly perceive a breach of TOS.

This is not good for the world.

Stop giving Google control of one of if not the most important communications identity in your life. I know gmail is cheap and easy, but theres a reason for that: you're giving up so much.

It used to be “do no evil”. Well google, evil is as evil does.

EU Chat Control will have an error budget of 10%

https://old.reddit.com/r/privacy/comments/voaicx/10_error_ra...

If, witnessing the kind of power Google wields unchecked, you haven't - at the very least - taken steps to regularly pull all of your data out of Google, because, oh, this kind of stuff just won't happen to me ... please reconsider.

Email account deletion and recovery should be regulated by law. Nowadays it's an essential service - just imagine that electricity provider, ISP or anyone else is doing the same without any period of notice, nor ability to dispute the problem with a company's representative.

The right to medical privacy should weigh much more than Google's right to spy on communication between you and your doctor.

This is crazy; I have pictures of my son taking his first bath or his first swim in a tub and I shudder to think that I have to delete those because some algorithm will brand me as a paedophile.

If my google account is blocked, I can’t even begin to imagine the extent of what would happen to my day to day.

Time to move out of gmail

Google, wether via algorithms or human reviewers looked at this kids groin from photos that the guy expected to be private. This is the weird questionable behavior that should be ilegal.

This is why I backup my phone manually, my Google account is not used for any email I really care about, and I don't use it to authenticate anywhere else, etc. Same with other big providers (and smaller ones though for softly different reasons: they may vanish or be eaten by the larger fish).

Having too much linked to one group, especially one famed for terrible user care when something goes wrong, is just asking for this sort of accident to happen, and for it to affect you as widely as possible.

So theoretically some bad actor can just send someone pictures of cp to a gmail (and maybe google workspace) address and get them banned and investigated? This cant be right ...

People simultaneously want very strict policing of online spaces AND are not willing to pay for them at all. What do they expect? It surprises me that anyone here is blaming google, how many thousands of dollars should they spend deciding whether to delete an account worth a few pennies a year? When NOT deleting it leads to big fines and their company being branded a pedophile enabler?

This is why in civilised societies there is no vigilante justice, but courts and lawyers. This is inconvenient, but unavoidable.

"rules are rules" and this is why we need to get off of privately owned infrastructure for anything that matter.

I have a suspicion that this is because their systems will still scan this guys' pictures and will still trip on this or another picture, and they can't or don't want to disable checks on his account... so that their conclusion is "fuck it, it's easier to keep him banned".

Is there any source in which countries this Google CSAM scanning works?

I wonder if he can ban anyone from Google by emailing them those pictures. Would AI detect those pictures as well? He should look for Google employees and ban them all

I have previously worried about this slightly when sharing videos of my daughter singing in the bath with my mum and my sisters. Pretty sad state of affairs.

At this point of publicity unban need to come with some "we are changing our policies" promise, which they are not willing to do maybe?

We need laws that bar these monopolies and near monopolies from being able to deplatform people.

Yes, good. Push this to the mainstream. Hopefully something good comes out of this.

I'll take the contrarian side. Why should Google allow underage nudes on their servers? There's so much wrong with this. Life's could be destroyed if their password gets cracked, the nudes leaked/sold. As a company I wouldn't touch this dumpsterfire with a stick.

Love it. More people scared of trusting large corporations will help everyone.

Okay, this convinces me to degoogle. What alternatives are there?

Isn't a father allowed to take photos of his son's groin even for non medical reasons?

Hmm, "The Circle" comes to mind. I wonder how many years we are away from this book becoming the reality.

Quit Google Now. Remember that you were always a bit stupid to choose Gmail and bite the bullet and get rid of it. If you are an Android user, I feel your pain.

The only thing I am a bit reluctant to throw away is my YT account. But if the content quality degradation keeps on going, it will eventually be quite easy to throw out.

Evil is as evil does.

TIL: upload child porn on a secret website, take someone's device, send said image and delete the chat, get his account banned with no possibility of reversal.

Thank God for AI, planting evidence and automatically ruining someone's livelyhood has never been so algorithmically efficient!

It’s stated clearly in the ToS Google will not stand for explicit images of a child. Google reserves the right to punish those who break the rules. It may not seem fair but it’s not worth the risk, it’s easier to just delete the user’s account and move on to the next user.

As a user, there is a dispute process but as a user I must obey the ToS if I don’t want my account to be deleted. If someone came into your place of business and began taking explicit photos of their child, for medical or any other reason, would you stand for it? Would you give them a second chance to explain themselves?

Google’s place of business is whatever you do on your phone and any content generated by it. You don’t own Android OS just by buying a phone the same way you don’t own a pizza shop just by buying a slice of pizza there.