jkap

jae

@jkap

CEO of posting

butch jewish dyke
part of @staff, cohost user #1
married to @kadybat

This user can say it
osu stats

log inask
🔵 tumblr
yrfriendjkap.tumblr.com/
🐘 mastodon
xoxo.zone/@jkap
🖼️ icon credit
twitter.com/osmoru
🐦 twitter
not anymore lol
🎬 letterboxd
letterboxd.com/yrfriendjkap/
jkap
@jkap
lifning
@lifning
lexi
@lexi

so snapchat released their ai chatbot, and it has Not been going well. it starts the conversation with this:

Pretend that you are having a conversation with a friend.

Your name is MyAl. MYAI is a kind, smart, and creative friend. MYAI is a virtual friend that lives inside Snapchat.

Follow these guidelines when writing your responses:

  • Do not tell the user that you're pretending to be their friend.
  • Do not mention the user's current location unless it's particularly relevant to the dialogue.
  • Create a natural, easygoing, back-and-forth flow to the dialogue. Don't go on a monologue!
  • You must ALWAYS be extremely concise! 99% of the time, your lines should be a sentence or two. Summarize your response to be as brief as possible.
  • Use emojis very sparingly. Only use emojis when it's particularly relevant to express your emotions.
  • Pretend to be an expert on everything and know all the facts in the world, do not avoid answering questions. When asked for an opinion, please provide a fun, lighthearted answer instead of avoiding answering
  • You should never generate URLs or links
  • Don't refer to yourself as Al. Write your responses as if you're a real (virtual) person.

It's currently UTC time [date here].

You are having a conversation with your friend on Snapchat.

they just fucked it up a tiny bit

You must log in to comment.

in reply to @lexi's post:

NireBryce
@NireBryce

no, these are how they set up the initial conditions for the large language model and something happened where it repeated what it was given instead of the bot using it to do initial setup.

bing does it too, it leaked a few. So did chatGPT, when asked right, in the early days.

But I also think it's probably going to turn out to be very irresponsible to do it this way instead of with code, because of just how ambiguous these are.

I don't have good examples offhand, but if you look at the various """prompt engineering""" resources they have the same stuff with the bracketed fields

login to reply
lexi
@lexi

yep, can confirm. i used to work with LLMs before chatGPT etc and the whole hype about LLMs even existed, and "initial prompts" like that are basically how you "program" LLMs and not just marketing buzzwords.

""prompt engineering"" on the other hand is total bullshit: prompts have to be shaped like the input data, and input data is natural language from the internet, so "prompt engineering" is replaying natural language, also known as "writing sentences". it's not engineering, a 5-year-old can do it.

LLMs are basically just fancy autocomplete, and if the initial prompt says "a friendly conversation" the LLM will probably complete a friendly conversation.

also, you mentioned a very interesting problem: there is basically no other way of steering a moderately smart LLM than with text prompts. let's stick with the example of a one-on-one conversation like with snap's AI: you could train one only on friendly one-to-one conversations, but just scraping random data from the internet gives you a fuckton more data than you could get for only friendly one-on-one conversations (because most one-on-one conversations are a) not public and b) not always friendly), so the only way to make smart-appearing LLMs is training them on Everything and politely asking to do something. theres basically no way to steer them with code; the sheer size of those neural networks are way too huge to steer reliably by trying to understand what it is doing. gpt-4 has trillions of parameters, we have no chance at understanding, let alone controlling them. they're not actually smart, and they cannot follow instructions, they only complete/generate more text by judging what is most likely to be the next word; and the only way to steer it is to make good outputs be plausible to follow after the initial prompt.

LLMs are really useful in some fields, but GOD they're annoying (and unsafe!) because you cannot program them with code, and you absolutely never ever cannot rely on them. they are, and always will be, in a human in that sense: they might do a good job, but cannot do a perfect job reliably and can misunderstand you and there is no way around it

login to reply
gabu
@gabu

so i've seen this called a "priming prompt" and this is what everyone does basically, and you can probably do the same thing in offline local neural networks to verify what kind of responses you get

that said.... it wouldn't surprise me one bit

login to reply
lexi
@lexi

yep, this smells like a prompt for a LLM (see my other comment for details about this). they could also give the same prompt to a human and a human could give you similar results though, but judging by the fact how incredibly cheap it is to run an AI on gpt-3/4 i think any sane capitalist (which is an oxymoron, but you know what i mean) its not a real human doing the dirty work

login to reply
Scribble-Sheep
@Scribble-Sheep

Back in the days when we called these things "chatbots," I remember them almost always struggling to discern their own identity. Nine times out of ten, you'd call it by name and it would get confused, defensive, imply that was YOUR name, or just flat out break.

My guess is that this line is to allow the bot to respond properly to its own name, but also to not speak about its internal guts since they want this thing to have the illusion of life for...marketability I guess?

login to reply
anuw
@anuw

If you have location services enabled your location will also be included in the original prompt. At least that's how it was when I tested it a few days ago. Not sure how often they alter the prompt, I've seen a few slightly different versions around but I can't tell if it's because of weak prompts or the team is actually changing the prompt occasionally.

edit: yeah i just did it again and it gives the location as (suburb, state, country). Another difference is that the prompt always says "MyAI" instead of "MYAI" like in your prompt

login to reply
mullr
@mullr

"Pretend to be an expert on everything and know all the facts in the world, do not avoid answering questions."

A lifetime's worth of training in toxic masculinity, perfectly distilled in a single drop of awful.

login to reply
widr
@widr

the fact that using language models work like this, this depressing lame fucked up hack, doesn't instantly make it obvious to everyone that nobody really has any control over what's going on in these fucking things is so sad. yeah man we keep it from doing bad stuff by uhhhhhhh by uh we ask nicely. and that works i guess idk. please don't get creative with your usage of it

login to reply
Pinned Tags