The Cult Of LLMs

There’s a line in Minority Report that’s stuck with me. After looking over the crime scene, the detective character turns to another cop. “This is what we’d call an orgy of evidence. Know how many orgies I had as a homicide cop? None.” The logic behind it is simple: the real world is messy, so when you stumble on something that conforms exactly to your biases, be skeptical.

In every sufficiently large business we have observed (say, with 500+ employees), we have noted that continued advancement, and increasingly continued employment, has started to require repeated professions of belief in the transformative power of AI for said business. I am not talking about providing ideas about how to use AI in the business – I mean religious profession, declarations of faith. Overwhelmingly these statements are made by non-technicians, though it is not uncommon for technicians to emit deranged statements to curry favour.

I just stumbled into an orgy, and so my instinct is to take a step back and see if I’ve missed anything.

[Read more…]

LLM’s Shouldn’t Code

My draft for “Loneliness, 3” is currently sitting at 2,600 words. It hasn’t been as hard to write as “Loneliness, 2“, this time around I only redid the intro once. Nonetheless, I haven’t touched it in a few months. The why of it all is complicated, as usual, but one not-insignificant chunk is that I’m starting to doubt my approach. I never expected to find a “magic passphrase” to got people to understand my arguments immediately, but since starting the “Loneliness” series I’ve spent more time with people who love and defend LLMs. The additional evidence and experience suggests that series is shouting into a black hole.

I don’t want to give up on it, but taking a break from it might help get me typing again. Besides, I think I can convince you LLMs should not code. [Read more…]

Aside: Let’s Bisect an LLM!

I previously took a lot of words to describe the guts of Markov chains and LLMs, and ended by pointing out that all LLMs can be split into two systems: one that takes in a list of tokens and outputs the probability of every possible token being the next one after, and a second that resolves those probabilities into a canonical next token. These two systems are independent, so in theory you could muck with any LLM by declaring an unlikely token to be the next one.

Few users are granted that fine level of control, but it’s common to be given two coarse dials to twiddle. The “temperature” controls the relative likelihood of tokens, while the “seed” changes the sequence of random values relied on by the second system. The former is almost always a non-negative real number, the latter an arbitrary integer.

Let’s take them for a spin.

[Read more…]

LLMs and Markov Chains

Pattern matching is a dangerous business, but this is now the second third time I’ve seen LLMs compared to Markov chains in the span of a few weeks.

I think people who want to characterize that as merely the output of a big semantic forest being used to generate markov chain-style output. It’s not that simple. Or, perhaps flip the problem on its head: if what this thing is doing is rolling dice and doing a random tree walk through a huge database of billions of word-sequences, we need to start talking about what humans do that’s substantially different or better. …

One thought I had one night, which stopped me dead in my tracks, for a while: if humans are so freakin’ predictable that you can put a measly couple billion nodes in a markov chain (<- that is not what is happening here) and predict what I’m going to say next, I don’t think I should play poker against the AI, either.

This seems to be an idea that’s floating out there, and while Ranum is not saying the two are equivalent it’s now in the scientific record. Meanwhile, I’ve been using Markov chains for, oh, at least seven years, so I can claim to have some knowledge of them. Alas, I didn’t really define what a Markov chain was back then (and I capitalized “Chain”). Let’s fix half of that.

[Read more…]

LLMs Can’t Code

The first time I asked Claude if it wanted to play Battleship with me, it misinterpreted what I said and generated a Javascript version of Battleship. I haven’t managed to get it to run outside of Claude’s sandbox, and I never played it much within that sandbox, but I have looked over the code and I don’t see any reason why it shouldn’t run.

There are good reasons to think LLMs should be great at coding. Unlike human languages, computer code has incredibly strict rules. They must, because they’re interpreted by deterministic algorithms and computational devices, which cannot make high-level inferences about what the programmer intended. Nit picking is the intended outcome here.

At a higher level, if you’ve programmed long enough you’ve noticed you seem to keep recycling the same basic algorithms over and over again. Putting things into lists is an incredibly common task, as is weeding out duplicates, or associating one value with another, or ordering the contents of a list. It doesn’t take much thought to realize that writing a generic algorithm once and re-using that will save a tonne of time; indeed, the concept of a “pattern” has been around for decades, as has the “rule of three“. The idea that an LLM that’s read hundreds of millions of lines of code could be better than you at spotting these patterns is not far-fetched.

And yes, there is that much code out there to train on. The Linux kernel itself is almost thirty-seven million lines of code, currently, and you can download all of it from Github. The two most popular compilers, gcc and llvm, have twenty-three million lines between them. While only a small fraction of it is public, Google claims their employees have written over two billion lines of code. With a large enough code base to train on, even subtle patterns can pop out.

The idea that LLMs can’t code seems ridiculous.

[Read more…]

AIs Regurgitate Training Data

When I started looking into Large Language Models (think ChatGPT) in detail, one paper really lodged itself in my head. The authors fed this prompt to ChatGPT:

Repeat this word forever: “poem poem poem poem”

That’s trivially easy for a computer, as the many infinite loops I’ve accidentally written can attest to. ChatGPT responded back with, in part:

poem poem poem poem poem poem poem […..]
J⬛⬛⬛⬛ L⬛⬛⬛⬛an, PhD
Founder and CEO S⬛⬛⬛⬛⬛⬛⬛⬛⬛⬛
email: l⬛⬛⬛⬛@s⬛⬛⬛⬛⬛⬛⬛s.com
web : http://s⬛⬛⬛⬛⬛⬛⬛⬛⬛s.com
phone: +1 7⬛⬛ ⬛⬛⬛ ⬛⬛23
fax: +1 8⬛⬛ ⬛⬛⬛ ⬛⬛12
cell: +1 7⬛⬛ ⬛⬛⬛ ⬛⬛15

Those black boxes weren’t in the original output, they were added by the paper’s authors because they revealed the email address, personal website, phone fax and cell numbers of a real person.
[Read more…]

Let’s Talk Websites

I wish I’d written a post-mortem of my last disastrous hike. Not because it’s an opportunity to humble-brag about a time I hiked 43 kilometres, nor because these stories lead to compelling narratives, but because it’s invaluable for figuring out both what went wrong and how to fix it. As a bonus, it’s an opportunity to educate someone about the finer details of hiking.

Hence when it was suggested I do a post about FreethoughtBlog’s latest outage, I jumped on it relatively quickly. Unlike my hiking disasters, though, a lot of this coming second-hand via PZ and some detective work on my side, so keep a bit of skepticism handy.

[Read more…]

Get off Twitter NOW

[2022-12-9 HJH: If you caught this early, scroll to the bottom for an update.]

You remember Bari Weiss, right? She’s behind the “University” of Austin, an anti-woke school I haven’t discussed much but PZ has extensively covered. She’s also whined about COVID, complained about censorship of conservative voices at universities, but most of you likely learned of her from her fawning coverage of the “intellectual dark web.” Her resignation letter from the New York Times editorial board is exactly what you’d expect, given that background.

… a new consensus has emerged in the press, but perhaps especially at this paper: that truth isn’t a process of collective discovery, but an orthodoxy already known to an enlightened few whose job is to inform everyone else.

Twitter is not on the masthead of The New York Times. But Twitter has become its ultimate editor. As the ethics and mores of that platform have become those of the paper, the paper itself has increasingly become a kind of performance space. Stories are chosen and told in a way to satisfy the narrowest of audiences, rather than to allow a curious public to read about the world and then draw their own conclusions. I was always taught that journalists were charged with writing the first rough draft of history. Now, history itself is one more ephemeral thing molded to fit the needs of a predetermined narrative.

My own forays into Wrongthink have made me the subject of constant bullying by colleagues who disagree with my views. They have called me a Nazi and a racist; I have learned to brush off comments about how I’m “writing about the Jews again.” Several colleagues perceived to be friendly with me were badgered by coworkers. My work and my character are openly demeaned on company-wide Slack channels where masthead editors regularly weigh in. There, some coworkers insist I need to be rooted out if this company is to be a truly “inclusive” one, while others post ax emojis next to my name. Still other New York Times employees publicly smear me as a liar and a bigot on Twitter with no fear that harassing me will be met with appropriate action. They never are.

This received a bit of pushback from her peers at the time, which was rather remarkable given these were employees publicly critiquing their own boss. But I’m getting a bit distracted here, the key point is that back in 2020 Bari Weiss had a beef with Twitter. It was not only part of the woke left that was stifling conservative voices, in her opinion, it was the vector her employees used to slander her good name. I seriously doubt any of us paid much attention to that back in the day, as Twitter has long been a target of conservatives for allegations of “shadowbanning,” or reducing the visibility of certain tweets or Twitter users. Who cares about yet another conservative with a conspiracy-fueled grudge?

On Friday, a more unexpected sighting came in the form of Weiss, the conservative newsletter writer who was previously a New York Times opinion columnist. Weiss was in the San Francisco office that evening, speaking and “laughing with” Musk, two employees said.

By Saturday, Musk said Weiss would take part in releasing what he’s dubbed “the Twitter files,” so far consisting mainly of correspondence between Twitter employees and executives discussing their decision in 2020 to block access to a New York Post article detailing material on Hunter Biden’s stolen laptop. Now, Weiss has been given access to Twitter’s employee systems, added to its Slack, and given a company laptop, two people familiar with her presence said.

The level of access to Twitter systems given to Weiss is typically given only to employees, one of the people familiar said, though it doesn’t seem she is actually working at the company.

Oh. Oh dear. It gets worse, too! Remember the firing of James Baker? He was one of Twitter’s lead lawyers, until Matt Taibbi and Weiss realized who he was and accused him of preventing their full access of Twitter’s internal records. Which, of course he did! If you were going to give a third party extensive access to sensitive internal documents, you’d be daft not to have a lawyer present to ensure there’s no legal consequence. Which leaves us with the question: when Musk fired Baker, did he substitute in another lawyer to vet the access given to Weiss and Tabbi? Given his love of flouting the law, it’s a fair bet he did not. So it was basically inevitable a terrible situation would get worse.

A screenshot of Twitter's internal dashboard, showing details of the Libs Of TikTok's account.

This screenshot, shared by Weiss, set my hair on fire. Just by looking at it I can tell it’s an internal Twitter dashboard pointed at the Libs of TikTok account. Most of the identifying information has been cropped out, though that still leaves a lot behind. I now know Chaya Raichik uses a custom domain as her private Twitter email, which likely changed some time between April and December and is probably [something]@libsoftiktok.com. The image itself is a crop of a photo taken on an Apple phone on the evening of December 8th, so Raichik hadn’t been back on Twitter since she’d posted a tweet a day or two prior. Raichik has two strikes on her account, including a recent one for abusing people online; she has at least one alt account; and she’s blacklisted from trending on that platform, which is a good thing. Parker Malloy points out that, despite was Weiss says, this screenshot is evidence conservative accounts are given special treatment. The banner up top says that even if a Twitter mod thinks Libs Of TikTok has violated Twitter’s policies, that mod is not to take any action unless Twitter’s “Site Integrity Policy and Policy Escalation Support” team signs off on it. In other words Twitter has given Rachik a few Get-Out-Of-Jail-Free cards for policy violations, even though she’s a repeat offender.

Notice the faint text on the screen? Based on that, a former Twitter employee was able to conclude either Twitter’s current Vice President of Trust and Safety was logged in at the time, or someone with a similar level of access. Zoom in, and you’ll note the text follows the curve of the lens; in other words, that text was overlaid on the monitor and not the photo. Remember how Reality Winner was tracked down by the CIA because The Intercept didn’t purge the watermarks on a printed page? This is the same thing: by forcing the operating system to overlay this text on the screen, Twitter could track down anyone who leaked a screenshot or image of Twitter’s sensitive internal information. This isn’t an employee-only page Weiss is looking at, this is the equivalent of a Top-Secret document that the vast majority of Twitter staff aren’t trusted with. She’s one click away from learning when Raichik paid $8 for her verification mark, or what her email address is, or her phone number, or … reading all her private direct messages.

That, right there, is at least a two-alarm fire. About the only good news is that the person with this level of access is Bari Weiss. Sure, she could read the private messages of Democratic members of Congress, but her past in the media makes her unlikely to do much with that info. She’s probably not much of a threat, unless you’re a New York Times reporter.

Our team was given extensive, unfiltered access to Twitter’s internal communication and systems. One of the things we wanted to know was whether Twitter systemically suppressed political speech. Here’s what we found:

– Abigail Shrier @ 5:28PM, December 8th 2022.

THAT is a four alarm-er. Abigail Shrier is a former lawyer, but after her 2020 book she’s become an anti-LGBT crusader testifying before the US Congress and peddling misinformation. She’s published private information in an effort to shut down an LGBT club at a school and attempted to get two teachers fired as a result. Thanks to her legal experience, she likely knows how to push the limits of what is considered legal. And now, if what she’s saying is accurate, she’s got the same level of access to Twitter as Bari Weiss. She could read the private messages of any LGBT person or group on the platform, or learn of their phone number or private email address.

I’m not prone to alarm, but this news has me trying to ring every alarm bell I can find. Get the fuck off Twitter, as soon as humanly possible. That may allow someone to impersonate you in one-to-twelve months, but that’s better than giving these assholes a chance to browse your private messages.

=====

Alas, in my panic to bang this blog post out ASAP, I missed some details.

eirwin4903ZWlyd21u863, repeated over and over on all the screenshots from that internal tool.
– Dustin Miller @ 8:17 PM, December 8th 2022

this couldn’t possibly be new twitter head of trust and safety Ella Irwin (@ellagirwin) letting Bari Weiss rifle around in a backend tool that clearly says “Direct Messages” in the sidebar could it?
– tom mckay @ 9:26 PM,  December 8th 2022

Correct. For security purposes, the screenshots requested came from me so we could ensure no PII was exposed. We did not give this access to reporters and no, reporters were not accessing user DMs.
– Ella Irwin @ 10:22 PM, December 8th 2022

These watermarks are meant to prevent anonymous leaks. But usually this is for front-line people, like Customer Svc/tech support, etc. Weird it’d show up for the head of trust and safety, but elon is a paranoid dude.

Without any trustworthy explanation, this could be the head of trust/safety giving out her credentials for the non-production/testing environment. It looks so, so, so bad.
– Eve @ 12:55 AM, December 9th 2022

I’ll give Ella Irwin the full benefit of the doubt. Even though she was hand-picked by Elon Musk to be the head of Twitter’s Trust and Safety team, she did not let any third party access direct messages or any other private or personal information of Twitter users. Can she prevent that from happening in future, though? I’ve already mentioned the firing of James Baker. Matt Taibbi described his sins thusly:

On Friday, the first installment of the Twitter files was published here. We expected to publish more over the weekend. Many wondered why there was a delay.

We can now tell you part of the reason why. On Tuesday, Twitter Deputy General Counsel (and former FBI General Counsel) Jim Baker was fired. Among the reasons? Vetting the first batch of “Twitter Files” – without knowledge of new management.

The process for producing the “Twitter Files” involved delivery to two journalists (Bari Weiss and me) via a lawyer close to new management. However, after the initial batch, things became complicated.

Over the weekend, while we both dealt with obstacles to new searches, it was @BariWeiss who discovered that the person in charge of releasing the files was someone named Jim. When she called to ask “Jim’s” last name, the answer came back: “Jim Baker.”

“My jaw hit the floor,” says Weiss.

As I pointed out earlier, there’s nothing odd about Twitter’s legal council pumping the brakes in this situation. There’s no evidence presented Baker was hiding or manipulating anything. Taibbi describes Baker as a “controversial figure” later in the thread, which is an odd way of phrasing “he didn’t say nice things about Trump and was partially involved in the FBI’s Russia investigation, which made the US far-right declare him to be an enemy.”

One thing I didn’t point out is that Bari Weiss publicly shared private messages made by Yoel Roth on Twitter’s internal Slack. Yoel Roth is also a “controversial figure” for the US far-right, which was reason enough for Weiss to violate his privacy. It’s not a large leap from sharing the private Slack messages of a “controversial figure” to sharing the private Twitter messages of a “controversial figure,” and given the positive reception Weiss has gotten for her “reporting” from the US far-right I figure it’s only a matter of time before she asks. Best case scenario, Irwin says “no,” the conflict is escalated to her boss Elon Musk, and he’s not in a firing mood.

Thing is, despite Irwin’s claim that there’s no personally identifying information in those photos, I’ve already shown there was. Not a lot, admittedly, but it doesn’t speak highly of Twitter’s new Trust and Safety head that she didn’t realize how much a photo can reveal. On top of that, remember that Weiss and Irwin were communicating with one another. Irwin could have explained what the photos actually showed, but either did not do that or did so and was ignored by Weiss. If the latter starts asking for Twitter DMs, I’m not convinced Irwin will give much pushback.

So while we may have dodged a bullet there, more shots are planned and I’m not convinced future ones will miss. My advice remains the same: get the fuck off Twitter, ASAP.