TLDR: Cryptograghy relies on the assumption of the difficulty of math problems. Cryptographic protocols are usually eroded slowly as analyses finds weaknesses. There are many other ways to attack security, which LLMs are middling at. But LLMs are solving some previously unsolved math problems. It's plausible that these are leveraged to weaken cryptography, and that might happen very fast. I've trained tons of NNs so here's how I'd do it if I had a grant.

cryptography only works if we can't solve it

tldr : Cryptography relies on the assumption that some math problems are hard to solve.

To put it very very simply, cryptography relies on the assumption that some math problems are difficult to solve. (Search "the hardness assumption of the discrete log" to learn more, or to make dick jokes). Think "y = mx + b, solve for x", except the combined effort of decades of mathematicians were unable to solve for X.

If you want to break cryptography, you have to:

• Solve for X. This is usually something the best mathematicians have tried to do for decades and have been unable to, so, good luck.

• Guess X. (But this might be something like a 1 in 2^128 shot, so even if you can guess 1 kirkillion times per second, it will take forever.) (This might mean solving the discrete log.)

• Find leaked information about X. This targets implementations, e.g. inferring power consumption of a CPU from the fan speed.

• Find someone who knows X (or the information X is being used to encrypt) and beat them with a wrench.

Different cryptographic constructions usually provide different promises, but they almost always take the form of "This can only be broken if someone knows X, or guesses X, but guessing that will take an expected 2^127 attempts, assuming the difficulty of the discrete log.

When cryptographic assumptions get weakened, usually it's not all-or-nothing. Maybe someone finds a way to leak a few bits here, a few bits there, or maybe weakening small parts of the construction. This is common, e.g. as happened with MD5. People started moving away from it a decade before a collision was actually found, and while MD5 does not give the guarantees you'd want from a hash function, you could still use it for something like PBKDF5. All this is to say, weakening a cryptographic assumption almost never completely destroys it.

But I want to also emphasize that nobody is "solving for X". Nobody is weakening cryptographic assumptions by trying to solve for the discrete log (or equivalent). Those are one of those hallowed math problems that nobody has ever solved, and nobody will solve in my lifeti-

Mathematicians are using AI to solve hallowed math problems that nobody has ever solved before

tldr : LLMs are solving math problems. This is indisputable and I'm sorry if I'm the one to break it to you. Unfortunately, all of our cybersecurity relies on some math problems being unsolvable.

It was never supposed to be this way. The lowly, natural-language, "neural" AI was never supposed to be solving math problems. That was the domain of the high, logic-based, "symbolic" AI. I guess it turns out a lot of math really does rely on the natural language context we couch it in.

If you're not aware, this is a place you "gotta hand it to em", LLMs are pushing the frontier of math. This isn't even controversial among mathematicians. New PhD students, postdocs I know, professors, math instructors, and even some of the most well-regarded minds in the field (Knuth, Tao) are all acknowledging it. I'm sorry if I'm the one to break this news to you.

This isn't as notable as it sounds. There are tons of unsolved problems in mathematics. (Very smart) children are solving them, 4chan users solve combinatorics problems in discussing anime, etc. It's not too much more impressive than the normal course of a math PhD, or a Futurama writer.

And LLMs are always, always producing bullshit. Ask any math PhD who's using AI and they'll tell you about how they keep getting generated text that's confidently wrong. This might be one field where AI is making people sharper, because you only ever get confidently-incorrect text when you're evaluating a colleagues papers or peer-reviewing for a conference.

But it's still notable that LLMs are good enough to produce proofs to math problems. There is no reason this can't weaken the math problems that underly all computer security.

This one simple trick to improve your models

tldr : Generating data to train on is very traditionally useful for neural networks.

A tangent.

One of the early tricks to improve the performance of any model is by pretraining it on any structured data. Like, at all.

Say you had a simple binary classification problem, e.g. whether or not to warm up a battery as part of a microgrid. Your input was a vector of size N and you output [0, 1] or [1, 0] (which is normal for reasons). If you only had a small bit of data, you might first

• Generate point clouds of two non-overlapping shapes in N dimensions, and feed that into your model (many points)

• Create a crude simulation of the inputs for your model (say, weather patterns, estimated people at home), and then train on that.

• and then train on your scarce data.

The ability to generate data of any structure (that fits your problem), and then to create simulated data, is very very useful!

This one simple trick to DESTROY THE WORLD

tldr: I'm exaggerating a bit. But if I were to pursue this avenue to attack cryptography I'd also generate a ton of math text programmatically.

One useful thing about math is that we've already spent a ton of time trying to encode the natural-language math into programmatic relations. That is hard manual labor, but the reverse is actually easy to do.

This means we can take (1) the corpus of human mathematical knowledge AND (2) generate even more of that (in a way that is completely accurate) as training data for our models.

I want to note, on 2, that I'd not use an LLM to generate math text. You can generate natural language from mathematical relations! Doing it that way might also yield some insights, but it'd mostly be a means to the end of creating good training data.

All of these LLMs being used for mathematical advances are general-purpose, and not for math specifically. They're being asked questions, which is dressed in layers of prompts, but the underlying txt2txt model makes no differentiation between its replies and your replies. It generates text, you add text onto it, and that's fed back into the start of the model.

The construction I am imagining is one in which the model is trained to independently generate correct text, and the model is not being used with a chat/prompt harness, but rather with raw access to the next tokens it's predicting. (I.e. "autocomplete", a-la what Botnik did in ~2018 to generate that Harry Potter text.)

I would expect this model to be way better for solving math problems, and I would expect using it this way to be way more productive.

This one simple trick is also super duper obvious

There is no way I'm the only person with this idea. A lot of people in deep-learning research have had their brains poisoned by LLMs-specifics, and have forgotten that these are all just seq2seq transformers where the sequences are natural language.

But not all of them! What I am proposing is super obvious stuff a first-year PhD student might have come up with in 2020.