There’s something about the internet mindset – the anonymity (or, I should say, “apparent anonymity”) on both sides, that brings out the worst in people. It seems to me as though half of the new internet entrepreneurs are trying to figure out how to screw the other half. And both are trying to screw their customers.
Back in the late 90s, I exchanged an odd series of emails with a fellow, who had tracked me down because he was wondering if I had any other pictures I could share with him, from the photoshoot I had done with his girlfriend. I was flummoxed; I didn’t know who he was talking about at all, so he sent me pictures of the model in question – pictures of two different girls, one photo I had taken, the other a photo lifted from a website run by a gothic seamstress (who I also recognized). His alleged girlfriend supposedly lived in New York but the model I had photographed lived in Virginia and the gothic seamstress in Texas. We swapped a few emails and he said, “I’ll get back to you.” Several months later, he did: he had confronted his online girlfriend and they had admitted that the pictures were “pictures of the person that I wish I was” and his girlfriend was not anyone remotely like who he thought he was talking to. For one thing, his girlfriend was a pretty creepy individual who was willing to chat up a lonely guy to try to get him emotionally involved so that they could get him to send money. The poor fellow was very upset and tried to offer me money to backtrack the pretender so he could “sort them out.” I declined.
This is nothing new in the history of internet scams. I’ve heard similar stories on podcasts – I just never expected that I’d get embroiled on the edge of such a situation. I was unsettled by how quickly his love turned to hatred and a desire for violent confrontation but that, too, is nothing new. You can’t really blame people for getting emotional, when you’re playing dishonestly with their emotions.
instagram ‘influencer’ Lil Miquela is a 3D rendered character
I think internet dating’s great if everyone involved is being honest. You can keep your search specific, if you want to; if you spend 20 minutes chatting up someone interesting at an event and they suddenly drop that they are a serious christian, or they wave some racist gang-sign, you have to scramble for a strategy to delicately get the fuck out of there. If you’ve seen Tim Minchin’s Storm you’ll know it’s a documentary of many people’s lives. [youtube] It’s the root of my social anxiety; having spent 16+ years on the conference circuit, I’ve gotten stuck at dinners next to moon landing conspiracy theorists, creationists, and one acupuncturist. If you’re looking for companionship, you can (in principle) filter out the people you simply don’t want to have to try to get along with. In terms of relationships, it’s nice to filter out those basic incompatibilities before they bite you. In the real world, this is a a real problem – I know one guy who was head over heels in love before his paramour told him that they were evangelical christian and wanted to produce a “quiver full” of white nationalist babies. Oops. If you’ve already allowed yourself to really care about that person, it’s kind of a bait-and-switch – you’ve suddenly got a bunch of decisions to make that you otherwise would have avoided.
The hypothetical scenarios that come to mind are the old Eliza program, and Lil’ Miquela. When Joseph Wiezenbaum wrote Eliza to emulate a therapist* one of the things he discovered was that some people would try to flirt with it. Reading the comments on Lil’ Miquela’s instagram, it appears to me that some people don’t realize right away that they are talking with a non-being. In the last year, whoever does the software behind Lil’ Miquela has added basic chatbotting to its instagram – if you leave a friendly message, you’ll get a friendly answer. There is a deeper question that I don’t want to go into, namely “what is Love?” but it seems plausible to me that someone out there who didn’t get the memo might fall in love with Lil’ Miquela. Let me dodge the question of semantics, there; should I say “think they have fallen in love with…” or “fool themselves…” is it possible to fall in love with someone who does not and never has existed?
These are all topics I don’t feel qualified to answer, but the profit-driven assholes of the internet, in their drive to “monetize” everything, are going to explore and see how they can make the world a worse place. This bugs the living fuck out of me, because it’s specifically acting against the customer who is paying for the service that is abusing their trust. This is just … weird. So, there are internet dating sites that are preparing to bulk up their membership with AI chatbots. [wp]
The AI software used to create such faces is freely available and improving rapidly, allowing small start-ups to easily create fakes that are so convincing they can fool the human eye. The systems train on massive databases of actual faces, then attempt to replicate their features in new designs.
One of the problems dating sites have is attracting enough members (generally, young, attractive women) that will talk to the paying members (generally men) and make them feel that they are getting their money’s worth. It’s been a few years since I was hanging out on OKCupid but at that time I remember it was pretty easy to spot the ringers – they’d bungie in and look at a profile, but never reply to a message, or they’d message something opaque like “Hi, hun!” and never reply any more than that. I know that some of those accounts were ringers because I recognized some of the girls in some of the photos; apparently whoever was creating the fake profiles had harvested some of the images from modeling portfolio sites I was familiar with.
That got so bad that the FTC, which notably is reluctant to do anything to protect customers, finally sued match.com/OKCupid for using fake profiles to attract customers to pay for membership: [ftc]
The Federal Trade Commission sued online dating service Match Group, Inc. (Match), the owner of Match.com, Tinder, OKCupid, PlentyOfFish, and other dating sites, alleging that the company used fake love interest advertisements to trick hundreds of thousands of consumers into purchasing paid subscriptions on Match.com.
The agency also alleges that Match has unfairly exposed consumers to the risk of fraud and engaged in other allegedly deceptive and unfair practices. For instance, the FTC alleges Match offered false promises of “guarantees,” failed to provide services to consumers who unsuccessfully disputed charges, and made it difficult for users to cancel their subscriptions.
The scam is pretty simple: you get a notice that “so-and-so has browsed your profile” and you click to check them out. Then, it says you must be a paid member to view another member’s profile details. You roll your eyes and think “I’m being scammed” but you pay anyway and – when you do a search in your area – the profile that you wanted to check out is nowhere to be found. They’ve gotten better and more subtle but the obvious next step is to outright create people, faces, and conversations.
Right now, there’s a lot of concern about “deep fakes” – videos in which neural network similarities are used to create as-trained images atop of existing people. A year ago, “deep fakes” were mostly garbage, in my opinion. But that was last year. They are getting better, faster.
I’ve more or less ignored “deep fakes” until now because they’ve been pretty bad, and mostly the people producing them are trying to produce “porn” of popular personas. As you can see, though, it’s getting pretty good. What are the implications?
Compare 80s Tomb Raider to Lil’ Miquela
I think it means that identity is dead. It’s been limping along since the 90s, when spam made email identity doubtful. Then, in the 00s, SS7-enabled robocalls began making telephonic identity doubtful. Image identity has been being slowly destroyed (Photoshop has been around for a long time!) but bulk, automated feature-editing means that motion fakes are possible, even good, and still fakes have been getting better and better. Rapid improvements in computer graphics (3D CG) have had an interesting side-effect of making UFO photo fakes obsolete. If you don’t believe me, try showing a young person some of the UFO photos from the 50s and 60s and there’s a good chance they won’t even realize it’s supposed to be a spaceship. It’s been possible to fake audio and voice effectively for some time – what’s left? Rather than retrenching and hoping to salvage identity by complaining about fakes, we need to reject identity entirely. “Need” may not be the right word: we may not have a choice; it’s going to be thrust upon us.
Back when I worked in computer security, identity was the lynchpin that everything revolves around. I mean, it still is, I just stopped caring. There is a huge amount of effort expended to identify who is doing what, and the next frontier is going to be the question of whether there is even a “who” in the first place. I knew a lot of the guys who worked on the public key infrastructure standards (an attempt to leverage encryption to build a hierarchy of identity attestations) but I consider all that work to be a failure because the systems that are expected to carry and assess that identity are not very reliable. [Robert Morris Sr, the NSA’s chief scientist in the late 80s commented, “using cryptography on the internet is like using an armored car to deliver a paper bag to a park bench”] I pointedly refused to rely on any identity attestations, but instead I would challenge people to remember an experience we had had together – otherwise I’d tell them “I don’t know you well enough to know who you are.” So I might get a text message saying “Marcus this is Fred X, this is my new cell phone number please update your contacts” and I’d reply, “authenticate by telling me where we last ate thai food.” The point of all of this is that, soon, we will be getting pestered by an infinite number of chatbots and it will be hard to tell if it’s a real person, or not.
When I get calls from the “police union fund” for donations, I usually reply “I don’t like cops.” The first generation robots would crash off their script and hang up. The last one I got, went into a repeat loop. I keep meaning to cancel my land-line, which I have had since the 1990s, because I have not gotten a meaningful phone call on it in nearly 2 years; my voice machine blinks endlessly with 200+ partial messages left by varying degrees of unsuccessful robot.
Another firm, the San Francisco-based start-up Rosebud AI, offers clients a chance at 25,000 photos of “AI-customized models of different ethnicities.” Company founder Lisha Li – who named it after an infinite-money cheat code she loved as a kid for the people-simulator game “The Sims” – said she first marketed the photos as a way for small businesses on online-shopping sites to invent stylish models without the need for pricey photography.
Her company’s source images came from online databases of free and uncopyrighted photos, and the system allows clients to easily superimpose different faces on a shifting set of bodies. She promotes the system as a powerful tool to augment photographers’ abilities, letting them easily tailor the models for a fashion shoot to the nationality or ethnicity of the viewer. “Face is a pain point that the technology can solve,” she said.
The point of all of this? Don’t fall in love, or open your heart or your wallet to someone you haven’t met in the flesh. Unless you are prepared to not care. Unless you are prepared to discard identity. If you do that, you can have all the friends you want.
“decisions to make that you otherwise would have avoided” – I’m not a fan of the moral systems I’ve run across so far in my life, which is why I describe myself as a moral nihilist. I do have a sort of moral precept that I’ll stand behind, which is that you should not unilaterally reduce another person’s choices. That seems to me to be a fair definition of reducing their liberty, i.e: subjugating them to some degree or another. This can be by commission or omission. So, if you’re exchanging phone numbers and personal details with someone and there’s some interest there, you are maneuvering them into a situation (therefore reducing their freedom of choice) if there’s something you’re choosing not to disclose.
* Revealing he had a really low opinion of therapists, in a very passive/aggressive way.
“is it possible to fall in love with someone who does not and never has existed?” – when I was a kid, I had a real crush on Madame Recamier; my parents had a lovely portrait of her, and I conceived the possibility that she might exist in the flesh, somewhere. I eventually figured out that she had existed, but had been dead for almost 200 years, and probably had bad teeth and body odor. But there was a time when I wanted to meet her.
“uncopyrighted face databases” – yeah, right. NBC reports: [NBC]
The latest company to enter this territory was IBM, which in January released a collection of nearly a million photos that were taken from the photo hosting site Flickr and coded to describe the subjects’ appearance. IBM promoted the collection to researchers as a progressive step toward reducing bias in facial recognition.
But some of the photographers whose images were included in IBM’s dataset were surprised and disconcerted when NBC News told them that their photographs had been annotated with details including facial geometry and skin tone and may be used to develop facial recognition algorithms.
The story of IBM’s dataset is pretty sad, but it’s not anywhere near as bad as the FBI’s classified database of photos of people. Apparently they’ve just shoveled every ID photo they could get their hands on into it, and now they have something that nobody really knows what it is.
My technique of asking “where did we last have thai lunch?” is subtle, and I think it offers an answer to the identity problem, since it’s hard to completely steal my experiences as they intersect with another person’s. One way of thinking of it is that I am leveraging continuity, which means also duration. For example, we recognize IBM as a long-term brand, compared with, say, a jillion internet sites. IBM relies on their brand but also on the continuity of brand identity. That’s also what credit reporting bureaus do: they report on the continuity of an individuals’ financial history. For many purposes, if you don’t have an ATM card, you don’t exist. We can identify fake humans by looking for mismatches in continuity; if something on the internet claims to be a 27 year-old human, they are lying if they do not have a financial history that is at least 10 years old. Email services like Google don’t understand this, but email accounts that have existed for 10 years don’t send spam (unless they are taken over, i.e.: their identity is compromised) with the coming wave of fakes, the one thing that will be hard to fake for a while is continuity of identity.
AFAIK there are scammy dating sites (not the big ones, probably) that hire people to run fake user accounts portraying as hot women. These are used for stringing along male users in expensive pay-per-message chats that go nowhere. Imagine the improved efficiency if/when these jobs can be automated with chatbots!
(Aside from financially and emotionally exploiting gullible lonely men, I presume this business model also teaches said men that “all women” are cruel bitches who would only ever string you along.)
Since love is an emotion and since emotional states are purely internal, with no necessary connection to external reality, I’d say yes.
Turns out, the emotional part of our brain (whatever that is, I’m not a neurologist) basically can’t tell the difference between real and imaginary. That’s why horror movies work. It’s also why we’re capable of planning for the future. The future doesn’t exist yet, but we still have distinct preferences for one outcome over another. Because we can imagine them.
Eighties Tomb Raider?
In the 1970s, the band The Who did a song “Pictures of Lilly.” Its about a boy who falls in love with a picture of the late Lilly Langtree from the 1800s, not knowing it’s not current. A fantasy relationship doesn’t need it to be real. If one is satisfied with a fantasy, that means not caring if it’s real. So AI makes it “better”, but it’s not a new thing.
Marcus must be getting old. Those graphics are practically photo-realistic by the standards of the ’80’s.
The guy sounds creepy too. Each person is free to chose which photos of themselves they want to share with a boyfriend. If a boyfriend wants more photos, he should ask his girlfriend. Contacting a photographer and asking for photos behind the girlfriend’s back is creepy.
What exactly is going on with this video? Was the face supposed to change at some point and show a different actor?
I couldn’t spot anything unusual about that video. I’m probably face blind anyway.
@Andreas Avester:
They replaced Jack Nicholson’s face and voice with Jim Carrey’s in an existing movie. It’s an impressive technology demo.
Ketil Tveiten@#9:
Marcus must be getting old
Definitely. Also, I tried to forget Tomb Raider existed.
I was sloppy. I remember 80s graphics that were better than Tomb Raider and missed my guess; I should have looked it up. (Compare the flight simulator on an SGI to Tomb Raider and there were 80s systems that were pretty good!)
Contacting a photographer and asking for photos behind the girlfriend’s back is creepy.
Definitely. The guy had serious issues with boundaries, and I think that probably also helped make it easier for the scammer to fool him.
I’m not sure if “scammer” is the right word; they were after the emotion, not money. Sort of an emotional vampire. A very disturbed person.
The whole experience was memorably weird.
It’s a strange world when you can no longer believe your own senses.
Online dating seems so fraught with lies that I wonder at its success. Perhaps the lonely are so desperate for love and companionship that they suspend their disbelief and chase hope.
My elderly, lonely father got tangled up with someone who was trying to get him to send them money to help pay their medical bills. I could easily see that some of the photos Dad was sent were photoshopped, but Dad didn’t want to hear it. He died (unexpectedly) without sending her money, but only after a lot of effort on my part. The person involved played to my dad’s insecurities and made him feel wanted and special. “She” presented as a pretty 40 something woman with outstanding bills from a car accident, and she kept telling my 80-year-old father he was handsome, sexy and desirable. She also sent him a whack of sexy (badly photoshopped) photos of herself. As you can imagine, reality was less appealing to him.
I would LURVE to see someone put The Orange Toddler-Tyrant into that screaming-Hitler scene from Downfall (with appropriate subtitles, of course).
I think Tomb Raider was mid-90s? At least, I remember a friend playing it on the first playstation in the mid 90s in college… I could never play it, with my complete lack of hand-eye coordination, but it was kinda a neat game to watch others play, and while it may not have been the first of it’s kind, I think it was a pretty noteworthy as far as 3-D game play and such. She looked pretty cool running and doing flips and dual wielding pistols and shooting wolves and such, but yeah, if you stopped her moving and zoomed in…
Anyways, I’m just sorta reminded of those orchids that trick bees/wasps/flies into pollinating them by resembling a female insect in a pose indicating it’s receptive to mating…
In case you haven’t already heard this particular case, Shankar Vedantam published a story on Hidden Brain about a pen pal scammer who made a pile of money off of gullible, lonely men via hand-written letters. In the end, a number of the scams’ victims defended the scammer. It’s defies explanation that a solid core of victims will reliably defend their scammer (Amway, Herbalife, Trump…).
johnson catman@#12:
I would LURVE to see someone put The Orange Toddler-Tyrant into that screaming-Hitler scene from Downfall
Brilliant!
“It was a PERFECT CALL!!”
It seems to me that, as Trump descends into his own downfall meltdown, his actual speeches start resembling the famous Downfall scene in style, and the more absurdist Downfall parody scripts in content.
If there’s one bright spot, it’s that people will go back to only trusting first hand experience. Our relationships will get more local and personal, less casual. If you can’t meet face to face or through a trusted common contact, don’t answer or give details. I won’t add anyone to my facebook friends list who doesn’t meet either criteria.
Such a world would suck for a person with niche interests who lives in a small town where very few people (if any) share their passion for something.
Moreover, if I could only make friends through common contacts in face to face meetings, I would have zero friends.
Besides, what about some LGBTQ person who lives in a very homophobic/transphobic town? The friends they make online probably would be the only ones that can make.