[GH-ISSUE #5346] Gemma2:27b start to output repetive trash after few generations #29107

Closed
opened 2026-04-22 07:45:33 -05:00 by GiteaMirror · 4 comments
Owner

Originally created by @Mikhael-Danilov on GitHub (Jun 27, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/5346

What is the issue?

Gemma2:27b start to output repetive trash after few generations

Session log

``` >>> Write a long story Please provide me with some information about the story you'd like me to write, such as:

  • Genre: What kind of story are you looking for? (e.g., fantasy, science fiction, romance, mystery, thriller)
  • Characters: Who are the main characters? What are they like?
  • Setting: Where and when does the story take place?
  • Plot: Give me a brief idea of what you want the story to be about. What is the main conflict? What are the
    characters' goals?
  • Tone: What kind of tone do you want the story to have? (e.g., serious, lighthearted, humorous, suspenseful)
  • Length: Do you have a preference for a shorter story or a longer one?

The more details you give me, the better I can craft a story to your liking. For example, you could tell me:

"Write a long story about a group of friends who discover a magical portal in their attic. The story should be a
fantasy adventure with a sense of wonder and a bit of humor. The friends are three teenagers, a bit awkward and unsure
of themselves, but eager to prove they can be brave. Give me a few ideas for the characters:

  • A shy, bookish girl who's secretly a coding prodigy: This could be a fun twist where her coding skills are
    essential for navigating the digital world of the portal.

  • A sarcastic and witty boy who's secretly a romantic: He might act tough and skeptical, but his true nature comes
    out in the adventure.

  • A kind-hearted, goofy boy who's surprisingly resourceful:

He's the one who's always making jokes and cracking wise, but he has a knack for finding solutions to problems in the
most unexpected ways.

  • A brave and adventurous girl who's always the leader: She's the first to jump into the unknown, but her courage
    often stems from a desire to protect her friends.

Let me know what kind of story you'd like to hear, and I'll do my best to weave a tale of teenage bravery and a
fantastical adventure!"

I'm excited to see what kind of story we can create together!

a fantasy story about hedgehog
Barnaby Bumblebottom wasn't your typical hedgehog. Oh, he had the prickly exterior, the love for juicy slugs, and the
tendency to curl into a ball at the slightest hint of danger. But while other hedgehogs in the Whispering Woods might
be content with a simple life of foraging and napping, Barnaby yearned for something more.

He dreamed of adventure, of venturing beyond the familiar hedges and brambles of his home, of seeing the world beyond
the burrow. But the truth was, Barnaby was terrified of anything new. He'd never even left his burrow to explore the
garden he lived in.

One day, while Barnaby was nervously nibbling on a particularly juicy earthworm, a gust of wind blew a shimmering,
iridescent butterfly into the garden. It landed gracefully on a dandelion clock, and a thought struck him: "This is it.
I'm going to finally get out of this boring burrow and see what's on the other side of that hedge."

He'd heard whispers from the elder hedgehogs about a magical world beyond the garden fence, a world where adventures
were plentiful and the tastiest beetles lived. But this was the first time Barnaby had seen it with his own eyes.

"A beetle with wings that sparkle? That's not something you see every day, is it?" squeaked Hazel, a young hedgehog who
was anything but shy.

Barnaby, startled by her voice, quickly uncurled and retreated behind a mushroom. "I don't know, I don't go outside,"
he mumbled, his spines tingling with embarrassment.

"But I heard you're a bit of a coding whiz, Hazel," a voice chuckled from behind the mushroom. "Why are you always
making me out to be the fearless adventurer, the one who'll code their way out of any danger, and the one who'll always
be the first to try something new," a voice chuckled. "I'm more of a 'let's stay in the burrow and eat' type of
hedgehog."

He was right. He was the first to create a program that could map the pathways of the Burrow Network, a secret web of
tunnels connecting the gardens of the human world to the world of the magical and the mundane.

Barnaby, the hedgehog, was a coding prodigy. He hadn't just mastered the usual programming languages; he'd cracked the
code of the human world, a world of wonder and danger to him, a shy, bookish creature who preferred the safety of his
cozy burrow.

He lived in a world of ones and zeroes, of code and algorithms. His days were spent curled up in the comfort of his
den, a world of circuits and algorithms, working on his latest project: a program to map the ancient pathways of the
Burrow Network.

Barnaby was convinced that he could access the 'Human World' through a code hidden in the leaves. "It's not hidden," he
said, "it's a map. And I'm going to write a program to find it. A program to unlock the secrets of the garden, the
mysteries of the world beyond."

Barnaby's eyes, usually filled with the gleam of a thousand lines of code, sparkled with the thrill of the unknown.
"It's a secret world," he said, "a world of magic and the mundane."

Barnaby's words, though spoken with a hint of bravado, were tinged with a wistful yearning. He'd always been fascinated
by the stories the older hedgehogs told about the world above.

He'd seen it, too. The world above, with its vibrant flowers and bustling bees, wasn't all sunshine and roses. He knew
that.

Barnaby, the hedgehog, was the key.

He knew the world above wasn't always safe. There were dangers, of course. But there was also beauty, adventure, and
the thrill of discovery.

He'd always wanted to experience it, but fear held him back. He'd always been a bit of a scaredy-cat.

He'd heard tales of the dangers that lurked above, tales of owls and hawks and hawks disguised as owls.

He imagined himself, a brave and adventurous hedgehog, venturing out into the world beyond his burrow, a world of
endless possibilities.

But the world above was a scary place for a hedgehog.

"I'm not sure I'm cut out for this," he muttered, glancing nervously at the shadows lengthening across the garden.

The other hedgehogs, who were used to his coding prowess, were not so familiar with his anxiety.

He'd seen them all, the squirrels and the sparrows, the butterflies and the bees, the rabbits and the birds, and the
flowers that bloomed with the most vibrant and beautiful colors.

But Barnaby's code, it was different.

He'd seen them, the rabbits and the birds, he'd seen them all. But the world of the den, with its safety and comfort,
was a world he knew.

He'd written a program, a map of the world, that connected them all.

He'd mapped the paths of the world beyond, the paths of the den that was.

He'd mapped the world of the den, the world of the human world.

He'd even written a program to map the interactions between them.

"But it's just a map, right?"

"It's a map," he said, "but it's more than that. It's a key to understanding the world, a way to bridge the gap between
the known and the unknown."

Barnaby knew he was just a shy, introverted hedgehog, but he was determined to be the one who broke the code.

He wanted to understand the world beyond the garden, the world of the humans.

He'd been working on a program for years, a program to understand the complex rules and regulations that governed the
world of humans.

"It's a challenge, for sure," the coder said.

The humans, Barnaby knew, were a mystery.

He'd heard stories about them, of course, but they were always about their bravery.

He'd seen their world up close, but he was a coder, not a warrior.

A creature of logic and code, not of battles and bloodshed.

But what if the world above was just a reflection of the world down?

What if the skills he used to navigate the world of the internet were the same skills that could help him navigate the
world of the humans?

He was a master of the code,

After all, he'd never needed a warrior's skills to survive the world above.

He'd figured out the pattern of the slugs,

The best time to catch them, the tastiest ones.

He'd discovered the secret paths they followed,

The algorithms that dictated their lives.

He'd even created a program

For his fellow hedgehogs, to help them navigate the world of the internet.

But the world above, was not the den, and he'd always been a little afraid of leaving.

He'd never been outside before.

But he knew

"The Lord of the Rings," J.R.R.

Tolkien's epic fantasy novel, the "Lord of the world's"

"The Hobbit" and "Lord of the Rings" are the story of a "

world's"

"

"The Lord of the Rings"

The Shire, the home of the hobbits, is a peaceful and idyllic land.

"The world's

of

"The Shire"

a.

"The"

The story "The Hobbit" and "The Lord of the Rings" are a

"The Shire"

"The Shire" is a place, and "The Lord of the Rings" is a "The Lord"

"The Shire" was a place, and it was a peaceful and idyllic.

"The Shire"

"The Hobbit"

The "of"

"

of

"The "

of the world.

"The Lord of the Rings"

"

"

"

of the world.

The hobbit

"S

The Shire

"

"

The Shire" is a place, the "Lord

The

"The

The Lord of the.

"The Shire"

a "The Shire"

"The

"The Hobbit"

"The Shire"

"The

The world

The

"

of

"

The

"The Hobbit"

"The

This.

"The

"

of the hobbits, and the Shire is a place.

"The Hobbit"

is a story about the "The

the world

a.

the world

is the world"

The Shire

"The

the world" is

a series of "The

the

The "The

"The

" is a place

is a

The, the " the world

T

The Hobbit

The Shire"

is a place of adventure.

"

The

"The

The Shire"

of

the world"

The "

"The

The Shire

The Shire

"

of

"The Lord of the Rings"

The.

a.

the, a

"The

"The Hobbit"

is a

"The

"

the "The Shire"

the "The Lord" the

"The Shire"

is a place

"The Hobbit"

of the "The

is a novel.

The

"The

T.

"The Shire"

The hobbit

"The Shire"

was a place to "The Shire

"The

is a

"The

is a

The

The

of the

.

"

"The

The Shire

"The

"The

of

"The

"

the"

of the world

.

"

the hobbits"

of the world"

The

The

a place

"

of the world

the "

"The

of the

a world

The "The

"

The Hobbit

"The

The Hobbit

The

"

is a

The Hobbit"

The Shire"

The^C

</p>
</details> 

### OS

Linux

### GPU

Nvidia

### CPU

Intel

### Ollama version

0.1.47
Originally created by @Mikhael-Danilov on GitHub (Jun 27, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/5346 ### What is the issue? Gemma2:27b start to output repetive trash after few generations <details><summary>Session log</summary> <p> ``` >>> Write a long story Please provide me with some information about the story you'd like me to write, such as: * **Genre:** What kind of story are you looking for? (e.g., fantasy, science fiction, romance, mystery, thriller) * **Characters:** Who are the main characters? What are they like? * **Setting:** Where and when does the story take place? * **Plot:** Give me a brief idea of what you want the story to be about. What is the main conflict? What are the characters' goals? * **Tone:** What kind of tone do you want the story to have? (e.g., serious, lighthearted, humorous, suspenseful) * **Length:** Do you have a preference for a shorter story or a longer one? The more details you give me, the better I can craft a story to your liking. For example, you could tell me: "Write a long story about a group of friends who discover a magical portal in their attic. The story should be a fantasy adventure with a sense of wonder and a bit of humor. The friends are three teenagers, a bit awkward and unsure of themselves, but eager to prove they can be brave. Give me a few ideas for the characters: * **A shy, bookish girl who's secretly a coding prodigy:** This could be a fun twist where her coding skills are essential for navigating the digital world of the portal. * **A sarcastic and witty boy who's secretly a romantic:** He might act tough and skeptical, but his true nature comes out in the adventure. * **A kind-hearted, goofy boy who's surprisingly resourceful:** He's the one who's always making jokes and cracking wise, but he has a knack for finding solutions to problems in the most unexpected ways. * **A brave and adventurous girl who's always the leader:** She's the first to jump into the unknown, but her courage often stems from a desire to protect her friends. Let me know what kind of story you'd like to hear, and I'll do my best to weave a tale of teenage bravery and a fantastical adventure!" I'm excited to see what kind of story we can create together! ✨ >>> a fantasy story about hedgehog Barnaby Bumblebottom wasn't your typical hedgehog. Oh, he had the prickly exterior, the love for juicy slugs, and the tendency to curl into a ball at the slightest hint of danger. But while other hedgehogs in the Whispering Woods might be content with a simple life of foraging and napping, Barnaby yearned for something more. He dreamed of adventure, of venturing beyond the familiar hedges and brambles of his home, of seeing the world beyond the burrow. But the truth was, Barnaby was terrified of anything new. He'd never even left his burrow to explore the garden he lived in. One day, while Barnaby was nervously nibbling on a particularly juicy earthworm, a gust of wind blew a shimmering, iridescent butterfly into the garden. It landed gracefully on a dandelion clock, and a thought struck him: "This is it. I'm going to finally get out of this boring burrow and see what's on the other side of that hedge." He'd heard whispers from the elder hedgehogs about a magical world beyond the garden fence, a world where adventures were plentiful and the tastiest beetles lived. But this was the first time Barnaby had seen it with his own eyes. "A beetle with wings that sparkle? That's not something you see every day, is it?" squeaked Hazel, a young hedgehog who was anything but shy. Barnaby, startled by her voice, quickly uncurled and retreated behind a mushroom. "I don't know, I don't go outside," he mumbled, his spines tingling with embarrassment. "But I heard you're a bit of a coding whiz, Hazel," a voice chuckled from behind the mushroom. "Why are you always making me out to be the fearless adventurer, the one who'll code their way out of any danger, and the one who'll always be the first to try something new," a voice chuckled. "I'm more of a 'let's stay in the burrow and eat' type of hedgehog." He was right. He was the first to create a program that could map the pathways of the Burrow Network, a secret web of tunnels connecting the gardens of the human world to the world of the magical and the mundane. Barnaby, the hedgehog, was a coding prodigy. He hadn't just mastered the usual programming languages; he'd cracked the code of the human world, a world of wonder and danger to him, a shy, bookish creature who preferred the safety of his cozy burrow. He lived in a world of ones and zeroes, of code and algorithms. His days were spent curled up in the comfort of his den, a world of circuits and algorithms, working on his latest project: a program to map the ancient pathways of the Burrow Network. Barnaby was convinced that he could access the 'Human World' through a code hidden in the leaves. "It's not hidden," he said, "it's a map. And I'm going to write a program to find it. A program to unlock the secrets of the garden, the mysteries of the world beyond." Barnaby's eyes, usually filled with the gleam of a thousand lines of code, sparkled with the thrill of the unknown. "It's a secret world," he said, "a world of magic and the mundane." Barnaby's words, though spoken with a hint of bravado, were tinged with a wistful yearning. He'd always been fascinated by the stories the older hedgehogs told about the world above. He'd seen it, too. The world above, with its vibrant flowers and bustling bees, wasn't all sunshine and roses. He knew that. Barnaby, the hedgehog, was the key. He knew the world above wasn't always safe. There were dangers, of course. But there was also beauty, adventure, and the thrill of discovery. He'd always wanted to experience it, but fear held him back. He'd always been a bit of a scaredy-cat. He'd heard tales of the dangers that lurked above, tales of owls and hawks and hawks disguised as owls. He imagined himself, a brave and adventurous hedgehog, venturing out into the world beyond his burrow, a world of endless possibilities. But the world above was a scary place for a hedgehog. "I'm not sure I'm cut out for this," he muttered, glancing nervously at the shadows lengthening across the garden. The other hedgehogs, who were used to his coding prowess, were not so familiar with his anxiety. He'd seen them all, the squirrels and the sparrows, the butterflies and the bees, the rabbits and the birds, and the flowers that bloomed with the most vibrant and beautiful colors. But Barnaby's code, it was different. He'd seen them, the rabbits and the birds, he'd seen them all. But the world of the den, with its safety and comfort, was a world he knew. He'd written a program, a map of the world, that connected them all. He'd mapped the paths of the world beyond, the paths of the den that was. He'd mapped the world of the den, the world of the human world. He'd even written a program to map the interactions between them. "But it's just a map, right?" "It's a map," he said, "but it's more than that. It's a key to understanding the world, a way to bridge the gap between the known and the unknown." Barnaby knew he was just a shy, introverted hedgehog, but he was determined to be the one who broke the code. He wanted to understand the world beyond the garden, the world of the humans. He'd been working on a program for years, a program to understand the complex rules and regulations that governed the world of humans. "It's a challenge, for sure," the coder said. The humans, Barnaby knew, were a mystery. He'd heard stories about them, of course, but they were always about their bravery. He'd seen their world up close, but he was a coder, not a warrior. A creature of logic and code, not of battles and bloodshed. But what if the world above was just a reflection of the world down? What if the skills he used to navigate the world of the internet were the same skills that could help him navigate the world of the humans? He was a master of the code, After all, he'd never needed a warrior's skills to survive the world above. He'd figured out the pattern of the slugs, The best time to catch them, the tastiest ones. He'd discovered the secret paths they followed, The algorithms that dictated their lives. He'd even created a program For his fellow hedgehogs, to help them navigate the world of the internet. But the world above, was not the den, and he'd always been a little afraid of leaving. He'd never been outside before. But he knew "The Lord of the Rings," J.R.R. Tolkien's epic fantasy novel, the "Lord of the world's" "The Hobbit" and "Lord of the Rings" are the story of a " world's" " "The Lord of the Rings" The Shire, the home of the hobbits, is a peaceful and idyllic land. "The world's of "The Shire" a. "The" The story "The Hobbit" and "The Lord of the Rings" are a "The Shire" "The Shire" is a place, and "The Lord of the Rings" is a "The Lord" "The Shire" was a place, and it was a peaceful and idyllic. "The Shire" "The Hobbit" The "of" " of "The " of the world. "The Lord of the Rings" " " " of the world. The hobbit "S The Shire " " The Shire" is a place, the "Lord The "The The Lord of the. "The Shire" a "The Shire" "The "The Hobbit" "The Shire" "The The world The " of " The "The Hobbit" "The This. "The " of the hobbits, and the Shire is a place. "The Hobbit" is a story about the "The the world a. the world is the world" The Shire "The the world" is a series of "The the The "The "The " is a place is a The, the " the world T The Hobbit The Shire" is a place of adventure. " The "The The Shire" of the world" The " "The The Shire The Shire " of "The Lord of the Rings" The. a. the, a "The "The Hobbit" is a "The " the "The Shire" the "The Lord" the "The Shire" is a place "The Hobbit" of the "The is a novel. The "The T. "The Shire" The hobbit "The Shire" was a place to "The Shire "The is a "The is a The The of the . " "The The Shire "The "The of "The " the" of the world . " the hobbits" of the world" The The a place " of the world the " "The of the a world The "The " The Hobbit "The The Hobbit The " is a The Hobbit" The Shire" The^C ``` </p> </details> ### OS Linux ### GPU Nvidia ### CPU Intel ### Ollama version 0.1.47
GiteaMirror added the bug label 2026-04-22 07:45:33 -05:00
Author
Owner

@arkohut commented on GitHub (Jun 28, 2024):

update of "PARAMETER repeat_penalty 1.1" may help.

<!-- gh-comment-id:2195992794 --> @arkohut commented on GitHub (Jun 28, 2024): update of "PARAMETER repeat_penalty 1.1" may help.
Author
Owner

@s-kostyaev commented on GitHub (Jun 28, 2024):

update of "PARAMETER repeat_penalty 1.1" may help.

Unfortunately it doesn't help.

<!-- gh-comment-id:2196506876 --> @s-kostyaev commented on GitHub (Jun 28, 2024): > update of "PARAMETER repeat_penalty 1.1" may help. Unfortunately it doesn't help.
Author
Owner

@bfroemel commented on GitHub (Jun 28, 2024):

I think this is a duplicate of #5341

<!-- gh-comment-id:2196822087 --> @bfroemel commented on GitHub (Jun 28, 2024): I think this is a duplicate of #5341
Author
Owner

@Mikhael-Danilov commented on GitHub (Jul 10, 2024):

Seems to be fixed in ollama 0.2.1

<!-- gh-comment-id:2221619459 --> @Mikhael-Danilov commented on GitHub (Jul 10, 2024): Seems to be fixed in ollama 0.2.1
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#29107