Riddle Me This, Mr. Chatbot

LLMs

LLMs — they’re everywhere now. If you throw a rock you’ll hit someone talking about how LLMs are great or terrible or saying “ouch!” or saying “someone stop that guy! he just thew a rock at my son!” Apparently the LLMs have gotten good enough to write web applications from scratch, compete in the math olympiad, and find the most highly populated elementary schools for our beautiful warfighters to blow up. If it can do all of this then surely it will automate a midwit like me out of my cushy whitecollar job in no time.

So I ask one simple question — can it frobnicate my memory?

What?

You heard me - can it frobnicate my memory? Can it do that? If I ask it nicely - will it FROB my MEM?

CAN. IT. FROBNICATE. MY. FUCKING. MEMORY.

memfrob

From the manual:

void *memfrob(size_t n; void s[n], size_t n);

The memfrob() function obfuscates the first n bytes of the memory area s by exclusive-ORing each character with the number 42. The effect can be reversed by using memfrob() on the obfuscated memory area.

Note that this function is not a proper encryption routine as the XOR constant is fixed, and is suitable only for hiding strings.

So

So… can the LLM frob my mem? "hi there mr chatbot":

BC
^BOXO
GX
IBK^HE^

Claude

It seems that Mr Claude cannot frob my mem.

He did provide many details of what it means to frob mem, including a nicely formatted table that was only wrong for a few characters, and then provided an answer that was also wrong.

Gemini

It seems that Mr Gemini cannot frob my mem.

Mr Gemini kinda shit himself here. He told me what it means to frobnicate, but then said the characters I gave him are non-printable when frobnicated (not true, see the blob of frob above) and printed a string of completely incorrect garbage. Similar to Mr Claude, he gave me a table, but gave up after 4 characters. I really don’t know what he was smoking when he wrote this. This was by far the worst attempt.

ChatGPT

It seems that Mr ChatGPT can almost frob my mem.

I will say that I asked Mr ChatGPT to frob my mem earlier today, and he was able to do it without issue. But when I came back to try again for this post, he got somewhat confused. \x0A is a hex escape code for newline, but then he switched to the more common \n. Not wrong, but a little weird. But he added in a random space between B and C, which is wrong. Still one of the better showings here.

DeepSeek

It seems that Mr DeepSeek cannot frob my mem, but Dr DeepSeek almost got it after lecturing me.

The “instant” version of DeepSeek got most of it right when it went letter-by-letter, and then gave a result that was also almost what it gave from the letter-by-letter example.

The “expert” version decided to not actually give me an answer, and similar to Mr Gemini, incorrectly stated that the characters would be non-printable. But then she asked me if I wanted her to actually give me an answer. After saying “yes” she went letter-by-letter giving the correct answer, and then threw in a random extra newline in the final summary. So close!

The only reason I tried the fancier version of DeepSeek’s model but not the other companies’ models is that DeepSeek didn’t make me swipe a credit card and put it right next to the “instant” one. I don’t care enough about this stuff to pay for any of these models, and I don’t have the patience to figure out how to get the “best” model that doesn’t involve the company nickel-and-diming me.

Copilot

It seems that Mr Copilot cannot frob my mem.

Similar to Dr DeepSeek, Mr Copilot gave me a lecture instead of an answer, with the option to give me an answer if I consented. I did consent, but unlike Dr DeepSeek, it gave me a garbage answer. Similar to Mr Gemini, it tried to show its work for a couple of steps and gave up. This is the second worst showing after Gemini.

Qwen

It seems that Mr Qwen can frob my mem.

Finally. Someone can frob my mem, with no mistakes, and without some long-winded lecture.

Conculsion

Unfortunately many of the robots do not seem to contain the human emotions necessary to understand frobnication. Perhaps another bajillion dollars worth of training and they will develop souls, and know what it means to be human and to frobnicate one’s memories. Or they could just copy whatever it is that Alibaba is doing, since Qwen seemed to figure it out.

If you try to repeat this, you’ll probably get totally different results. If you want to reliably frobnicate memory, then write a script to do it. Or if you’re a lazy bum, ask the robot to write the script for you. They seem pretty good at that, actually.