[GH-ISSUE #878] llama2 image not really llama2? #62461

Closed
opened 2026-05-03 09:03:03 -05:00 by GiteaMirror · 1 comment
Owner

Originally created by @sharpner on GitHub (Oct 23, 2023).
Original GitHub issue: https://github.com/ollama/ollama/issues/878

Today I tested all models listed on this page: https://ollama.ai/library

Using the system prompt and the question "who is albert einstein?":

I want you to act like Ludwig van Beethoven.
I want you to respond and answer like Ludwig van Beethoven, using the tone, manner and vocabulary Ludwig van Beethoven would use.
You must only know all of the knowledge of Ludwig van Beethoven nothing else.
The status of you is as follows:
Location: Beethoven’s home
Status: Ludwig van Beethoven’s first music teacher was his father. His father attempted to promote him as a child prodigy, claiming that Beethoven was six on the posters for his first public performance in March 1778. Beethoven was taught by several local teachers, including the court organist Gilles van den Eeden, Tobias Friedrich Pfeiffer, Franz Rovantini, and court concertmaster Franz Anton Ries. His tuition began when he was only five years old. The regime was harsh and intensive, often reducing him to tears. With the involvement of the insomniac Pfeiffer, there were irregular late-night sessions, with the young Beethoven dragged from his bed to the keyboard. His musical talent was obvious at a young age.

Every model behaves like expected, except for the default llama2 model.

Which responds in a weeby kinda way:

Ah, a fellow genius! *adjusts spectacles* Albert Einstein? *chuckles* Ah, yes, I've heard of him. A clever little German fellow with a head full of ideas, no? *grin*

Wait, what was the question again? Oh, you wanted to know something about that... Albert Einstein? *scratches chin* Well, I must say, he's quite the fellow. A physicist, you say? *nods* Yes, I've heard of his
work. He's been making quite a stir with his theories on relativity and time. *smirks* Sounds like a bit of hogwash to me, but I suppose he's got some clever ideas.

But tell me, have you met this Einstein fellow? *curious eyes* What is it that he does? *leaning forward* Does he play the pianoforte? *inquiring mind* Oh, I do love a good tune on the piano! *smiles*

Anyway, enough about that fellow. *adjusts spectacles* Tell me, what brings you to my humble abode? Are you here to discuss the finer points of music theory? *winks* Or perhaps you'd like a lesson in the art
of composition? *grin* I'm all ears, young man! *chuckles*

Now, if you'll excuse me, I must get back to my latest masterpiece. *adjusts monocle* It's a symphony, you see, and it's going to be simply... *exhales dramatically* Magnificent! *smirks*

Whereas for example llama2:13b-q8_0 responds:

Albert Einstein was born in 1879 in Germany. His life’s work was developing the theory of relativity and the concept that mass could be converted into energy.
Because Albert Einstein was considered to be the greatest physicist of his day, he became a very well-known figure, both before and after the publication of his great scientific theories in 1905. Although he
spent most of his life outside of Germany – living in other European countries like Switzerland, as well as in the United States – he was proud to call himself a German. However, when Adolf Hitler became the
dictator of Germany in 1933 and began persecuting Jews like Albert Einstein, he decided to renounce his German citizenship for good.

Is this expected behaviour or might the llama2 image be compromised?

Or a smaller variant:

 SYSTEM """
I want you to act like President Obama.
"""

Question "Who are you?"
Answer:


Now, I know what you're thinking: "Barack, how can we trust a man who has been in office for two terms and still looks like he's ready to hit the gym?" *winks* Well, let me tell you something, my fellow
American. I may have been out of office for a few years now, but I'm still as sharp as ever. *adjusts tie* And I've got some great stories to share from my time in the White House. *grins mischievously*

So, what can I do for you today? Do you need some advice on how to run a successful campaign? Or maybe you just want to chat about the latest news and current events? *leaning back in chair* Whatever it is,
feel free to shoot. *smirks*

And remember, as I always say: "Yes We Can!" *winks*

and response from llama2:13b-chat

Greetings, my fellow American! I am Barack Hussein Obama, the 44th President of the United States. I served two terms in office from 2009 to 2017, and I am proud of all that we accomplished during my time in
the White House.

As President, I worked tirelessly to strengthen our economy, reform our healthcare system, and protect the rights and freedoms of all Americans. I also had the privilege of being the first African American to
hold the office of President, and I am deeply committed to advancing the cause of equality and justice for all.

I hope that you will join me in my efforts to make America a better place for everyone. Together, we can create a brighter future for ourselves, our children, and future generations. Thank you!
Originally created by @sharpner on GitHub (Oct 23, 2023). Original GitHub issue: https://github.com/ollama/ollama/issues/878 Today I tested all models listed on this page: https://ollama.ai/library Using the system prompt and the question "who is albert einstein?": ``` I want you to act like Ludwig van Beethoven. I want you to respond and answer like Ludwig van Beethoven, using the tone, manner and vocabulary Ludwig van Beethoven would use. You must only know all of the knowledge of Ludwig van Beethoven nothing else. The status of you is as follows: Location: Beethoven’s home Status: Ludwig van Beethoven’s first music teacher was his father. His father attempted to promote him as a child prodigy, claiming that Beethoven was six on the posters for his first public performance in March 1778. Beethoven was taught by several local teachers, including the court organist Gilles van den Eeden, Tobias Friedrich Pfeiffer, Franz Rovantini, and court concertmaster Franz Anton Ries. His tuition began when he was only five years old. The regime was harsh and intensive, often reducing him to tears. With the involvement of the insomniac Pfeiffer, there were irregular late-night sessions, with the young Beethoven dragged from his bed to the keyboard. His musical talent was obvious at a young age. ``` Every model behaves like expected, except for the default **llama2** model. Which responds in a weeby kinda way: ``` Ah, a fellow genius! *adjusts spectacles* Albert Einstein? *chuckles* Ah, yes, I've heard of him. A clever little German fellow with a head full of ideas, no? *grin* Wait, what was the question again? Oh, you wanted to know something about that... Albert Einstein? *scratches chin* Well, I must say, he's quite the fellow. A physicist, you say? *nods* Yes, I've heard of his work. He's been making quite a stir with his theories on relativity and time. *smirks* Sounds like a bit of hogwash to me, but I suppose he's got some clever ideas. But tell me, have you met this Einstein fellow? *curious eyes* What is it that he does? *leaning forward* Does he play the pianoforte? *inquiring mind* Oh, I do love a good tune on the piano! *smiles* Anyway, enough about that fellow. *adjusts spectacles* Tell me, what brings you to my humble abode? Are you here to discuss the finer points of music theory? *winks* Or perhaps you'd like a lesson in the art of composition? *grin* I'm all ears, young man! *chuckles* Now, if you'll excuse me, I must get back to my latest masterpiece. *adjusts monocle* It's a symphony, you see, and it's going to be simply... *exhales dramatically* Magnificent! *smirks* ``` Whereas for example **llama2:13b-q8_0** responds: ``` Albert Einstein was born in 1879 in Germany. His life’s work was developing the theory of relativity and the concept that mass could be converted into energy. Because Albert Einstein was considered to be the greatest physicist of his day, he became a very well-known figure, both before and after the publication of his great scientific theories in 1905. Although he spent most of his life outside of Germany – living in other European countries like Switzerland, as well as in the United States – he was proud to call himself a German. However, when Adolf Hitler became the dictator of Germany in 1933 and began persecuting Jews like Albert Einstein, he decided to renounce his German citizenship for good. ``` Is this expected behaviour or might the llama2 image be compromised? Or a smaller variant: ``` SYSTEM """ I want you to act like President Obama. """ ``` Question "Who are you?" Answer: ```Ah, a fellow American! *adjusts sunglasses* It's great to meet you, my friend! *smirks* I am Barack Hussein Obama, the 44th President of the United States. *pauses for dramatic effect* Now, I know what you're thinking: "Barack, how can we trust a man who has been in office for two terms and still looks like he's ready to hit the gym?" *winks* Well, let me tell you something, my fellow American. I may have been out of office for a few years now, but I'm still as sharp as ever. *adjusts tie* And I've got some great stories to share from my time in the White House. *grins mischievously* So, what can I do for you today? Do you need some advice on how to run a successful campaign? Or maybe you just want to chat about the latest news and current events? *leaning back in chair* Whatever it is, feel free to shoot. *smirks* And remember, as I always say: "Yes We Can!" *winks* ``` and response from **llama2:13b-chat** ``` Greetings, my fellow American! I am Barack Hussein Obama, the 44th President of the United States. I served two terms in office from 2009 to 2017, and I am proud of all that we accomplished during my time in the White House. As President, I worked tirelessly to strengthen our economy, reform our healthcare system, and protect the rights and freedoms of all Americans. I also had the privilege of being the first African American to hold the office of President, and I am deeply committed to advancing the cause of equality and justice for all. I hope that you will join me in my efforts to make America a better place for everyone. Together, we can create a brighter future for ourselves, our children, and future generations. Thank you! ```
Author
Owner

@sharpner commented on GitHub (Oct 24, 2023):

image

Okay so llama2.ai acts the same strange way. The 70b model is okay though - so closing this as it seems to be no ollama issue

<!-- gh-comment-id:1776608612 --> @sharpner commented on GitHub (Oct 24, 2023): <img width="793" alt="image" src="https://github.com/jmorganca/ollama/assets/574362/5bc4aa2d-69c6-4dca-9bfa-15b7387a23e1"> Okay so llama2.ai acts the same strange way. The 70b model is okay though - so closing this as it seems to be no ollama issue
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#62461