[GH-ISSUE #515] Where did you you get the original model source as its output contains weird words/phrases #26000

Closed
opened 2026-04-22 01:52:58 -05:00 by GiteaMirror · 6 comments
Owner

Originally created by @alexcardo on GitHub (Sep 12, 2023).
Original GitHub issue: https://github.com/ollama/ollama/issues/515

Hello, I'm very appreciate of your job. Yet, there is some weird thing I stumbled upon while playing with your model. When I type ollama run llama2 in my terminal and provide a prompt, I see that the second sentence always starts with a small letter (not capitalized). Secondly, in 70% cases, I see a Polish word in the output.

Here are the examples below:

alex@M1 ~ % ollama run llama2

Write me a short description of the Chanel brand
Chanel is a luxury fashion brand founded by Coco Chanel in 1910. surely one of the most iconic and influential fashion brands in the world. Chanel is known for its timeless, sophisticated designs that blend simplicity, elegance, and French chic.
...
Why starting from the non-capitalized letter?!?!

alex@M1 ~ % ollama run llama2

Write me a short bio of Vladimir Putin
Vladimir Putin is the current President of Russia, a position he has held since 2012. sierpiona 1999. Prior to this, he served as the Prime Minister of Russia from 1999 to 2000 and then again from 2008 to 2012. Putin was born on October 7, 1952, in Leningrad, Soviet Union (now Saint Petersburg, Russia).
....
What is this: "sierpiona 1999"?!?!?!

I have also seen some really Polish words... Unfortunately, I can't found them as I closed the terminal.

Looks like you're using not the original LLAMA 2 model as well as not even the quantized model by llama.cpp but your own fine-tuned model instead.

I've been trying the same prompts using the original LLAMA 2 weights from META as well as quantized ones from llama.cpp.

Please clarify the situation. Thank you!

P.s.: Found it...

alex@M1 ~ % ollama run llama2

Write me a 100-word description of Harmonic Arts Mushroom Capsules
Sure! Here's a 100-word description of Harmonic Arts Mushroom Capsules:
живело in a vegan, gluten-free capsule, Harmonic Arts Mushroom Capsules offer a potent blend of medicinal mushrooms, including Lion's Mane, Reishi, and Chaga. Each capsule is formulated to provide a consistent and reliable dose of these powerful mushrooms, promoting overall wellness and vitality. With no fillers or additives, Harmonic Arts Mushroom Capsules are a pure and effective way to incorporate the benefits of medicinal mushrooms into your daily routine.
...

It contains a non-English word: живело

I don't even know what does it mean. It's a 100% NON original LLAMA 2 model... (((


I like it very much how your code perform, please allow us to load the original LLAMA 2 model or its quantized version or whatsoever. Otherwise, it's useless as the model provided by contains crucial mistakes.

Originally created by @alexcardo on GitHub (Sep 12, 2023). Original GitHub issue: https://github.com/ollama/ollama/issues/515 Hello, I'm very appreciate of your job. Yet, there is some weird thing I stumbled upon while playing with your model. When I type ollama run llama2 in my terminal and provide a prompt, I see that the second sentence always starts with a small letter (not capitalized). Secondly, in 70% cases, I see a Polish word in the output. Here are the examples below: alex@M1 ~ % ollama run llama2 >>> Write me a short description of the Chanel brand Chanel is a luxury fashion brand founded by Coco Chanel in 1910. surely one of the most iconic and influential fashion brands in the world. Chanel is known for its timeless, sophisticated designs that blend simplicity, elegance, and French chic. ... Why starting from the non-capitalized letter?!?! alex@M1 ~ % ollama run llama2 >>> Write me a short bio of Vladimir Putin Vladimir Putin is the current President of Russia, a position he has held since 2012. sierpiona 1999. Prior to this, he served as the Prime Minister of Russia from 1999 to 2000 and then again from 2008 to 2012. Putin was born on October 7, 1952, in Leningrad, Soviet Union (now Saint Petersburg, Russia). .... What is this: "sierpiona 1999"?!?!?! I have also seen some really Polish words... Unfortunately, I can't found them as I closed the terminal. Looks like you're using not the original LLAMA 2 model as well as not even the quantized model by llama.cpp but your own fine-tuned model instead. I've been trying the same prompts using the original LLAMA 2 weights from META as well as quantized ones from llama.cpp. Please clarify the situation. Thank you! P.s.: Found it... alex@M1 ~ % ollama run llama2 >>> Write me a 100-word description of Harmonic Arts Mushroom Capsules Sure! Here's a 100-word description of Harmonic Arts Mushroom Capsules: живело in a vegan, gluten-free capsule, Harmonic Arts Mushroom Capsules offer a potent blend of medicinal mushrooms, including Lion's Mane, Reishi, and Chaga. Each capsule is formulated to provide a consistent and reliable dose of these powerful mushrooms, promoting overall wellness and vitality. With no fillers or additives, Harmonic Arts Mushroom Capsules are a pure and effective way to incorporate the benefits of medicinal mushrooms into your daily routine. ... It contains a non-English word: живело I don't even know what does it mean. It's a 100% NON original LLAMA 2 model... ((( ---- I like it very much how your code perform, please allow us to load the original LLAMA 2 model or its quantized version or whatsoever. Otherwise, it's useless as the model provided by contains crucial mistakes.
Author
Owner

@technovangelist commented on GitHub (Sep 13, 2023):

The Llama 2 model is the version quantized by The Bloke on HuggingFace. He got his source direct from Meta. For all the models on Ollama, you can go to the library page and scroll down, we show our sources for each one. (https://ollama.ai/library/llama2).

For instance, here is the 7b parameter model source: https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML

For more recent models we have been doing the quantizations ourselves.

And yes, I agree that sometimes the original Llama2 model does output some weird things.

<!-- gh-comment-id:1718272327 --> @technovangelist commented on GitHub (Sep 13, 2023): The Llama 2 model is the version quantized by The Bloke on HuggingFace. He got his source direct from Meta. For all the models on Ollama, you can go to the library page and scroll down, we show our sources for each one. (https://ollama.ai/library/llama2). For instance, here is the 7b parameter model source: https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML For more recent models we have been doing the quantizations ourselves. And yes, I agree that sometimes the original Llama2 model does output some weird things.
Author
Owner

@mxyng commented on GitHub (Sep 13, 2023):

You can verify this yourself. Quantized model weights are content addressable in both Hugging Face (through Git LFS) and Ollama:

TheBloke/Llama-2-7B-Chat-GGML/q4_0.bin

image

$ ollama pull llama2:7b-chat-q4_0
$ jq -r '.layers[] | select(.mediaType == "application/vnd.ollama.image.model") | .digest, .size' ~/.ollama/models/manifests/registry.ollama.ai/library/llama2/7b-chat-q4_0
sha256:8daa9615cce30c259a9555b1cc250d461d1bc69980a274b44d7eda0be78076d8
3791725184
<!-- gh-comment-id:1718289995 --> @mxyng commented on GitHub (Sep 13, 2023): You can verify this yourself. Quantized model weights are content addressable in both Hugging Face (through Git LFS) and Ollama: [TheBloke/Llama-2-7B-Chat-GGML/q4_0.bin](https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/blob/main/llama-2-7b-chat.ggmlv3.q4_0.bin) ![image](https://github.com/jmorganca/ollama/assets/2372640/0333f2b0-12fa-45ea-8445-837e48267cd4) ``` $ ollama pull llama2:7b-chat-q4_0 $ jq -r '.layers[] | select(.mediaType == "application/vnd.ollama.image.model") | .digest, .size' ~/.ollama/models/manifests/registry.ollama.ai/library/llama2/7b-chat-q4_0 sha256:8daa9615cce30c259a9555b1cc250d461d1bc69980a274b44d7eda0be78076d8 3791725184 ```
Author
Owner

@alexcardo commented on GitHub (Sep 14, 2023):

And yes, I agree that sometimes the original Llama2 model does output some weird things.

I have the original LLAMA 2 model as well as this https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML model. Both of them don't behave like ollama.

In my case, ollama is the only wrapper that works correctly as a wrapper and thereby can be used in "production". However, as the model output is always wrong, I'm forced to find another solution. llama-cpp-python has awful API, and I was hoping to use your implementation. Unfortunately, the output is always wrong. I suppose there this IS an error and other users will soon discover it.

I just suspect that something gone wrong during the process of quantization of the original model, as no matter how I tried to use is I was unable to get the results comparable to the output of the ollama model.

In all the rest, you've made a good implementation, but I still can't realize why you don't allow people to pick their models and use different download folder then .ollama. It would be a perfect solution just to load the model from the place where I want instead of storing it in a hidden folder.

It just looks a bit weird when you're making something really useful and exciting and providing some limited functionality !??!!

<!-- gh-comment-id:1718602743 --> @alexcardo commented on GitHub (Sep 14, 2023): > And yes, I agree that sometimes the original Llama2 model does output some weird things. I have the original LLAMA 2 model as well as this https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML model. Both of them don't behave like ollama. In my case, ollama is the only wrapper that works correctly as a wrapper and thereby can be used in "production". However, as the model output is always wrong, I'm forced to find another solution. llama-cpp-python has awful API, and I was hoping to use your implementation. Unfortunately, the output is always wrong. I suppose there this IS an error and other users will soon discover it. I just suspect that something gone wrong during the process of quantization of the original model, as no matter how I tried to use is I was unable to get the results comparable to the output of the ollama model. In all the rest, you've made a good implementation, but I still can't realize why you don't allow people to pick their models and use different download folder then .ollama. It would be a perfect solution just to load the model from the place where I want instead of storing it in a hidden folder. It just looks a bit weird when you're making something really useful and exciting and providing some limited functionality !??!!
Author
Owner

@mxyng commented on GitHub (Sep 14, 2023):

You're more than welcome to bring your own GGML and (starting 0.0.19) GGUF models. Importing it into ollama is simple:

FROM /path/to/model/bin

# Optional template. Defaults to `{{ .Prompt }}`
TEMPLATE <add your own templating>
SYSTEM <add your own system prompt>

See https://github.com/jmorganca/ollama/blob/main/docs/modelfile.md for more complete documentation on how to use the Modelfile.

Once you have this file, create an ollama model using ollama create <name>

<!-- gh-comment-id:1719825255 --> @mxyng commented on GitHub (Sep 14, 2023): You're more than welcome to bring your own GGML and (starting 0.0.19) GGUF models. Importing it into ollama is simple: ``` FROM /path/to/model/bin # Optional template. Defaults to `{{ .Prompt }}` TEMPLATE <add your own templating> SYSTEM <add your own system prompt> ``` See https://github.com/jmorganca/ollama/blob/main/docs/modelfile.md for more complete documentation on how to use the Modelfile. Once you have this file, create an ollama model using `ollama create <name>`
Author
Owner

@alexcardo commented on GitHub (Sep 15, 2023):

Thank you all that pointing me out the way on how to use my own quantized model. It works. However, looks like the issue I'm experiencing doesn't relate to the model itself. It likely relates to the wrapper. Even while using my quantized model via the ollama wrapper I see some weird useless output:

alex@NLDW4-5-20-11:~/ai/ollama$ ./ollama run ll2o

Write me a short description of the Adidas brand

zekero (16) April 24, 2020 (updated)
Adidas is a German multinational sportswear company that was founded in 1948 by Adolf Dassler. The brand is known for its high-quality athletic wear and footwear, including shoes, cleats, and other performance-enhancing gear. Adidas is particularly famous for its iconic three-striped logo and has been associated with some of the most successful athletes in sports history, including Lionel Messi, Kareem Abdul-Jabbar, and Tiger Woods. The brand has a strong focus on innovation and technology, and its products are designed to provide optimal performance and comfort for athletes at all levels.

What is this: "zekero (16) April 24, 2020 (updated)"... it's a garbage piece of the output. There is possible some simple solution. If you know how to get rid of this, I would appreciate any advice.

Thank you all.

<!-- gh-comment-id:1720970301 --> @alexcardo commented on GitHub (Sep 15, 2023): Thank you all that pointing me out the way on how to use my own quantized model. It works. However, looks like the issue I'm experiencing doesn't relate to the model itself. It likely relates to the wrapper. Even while using my quantized model via the ollama wrapper I see some weird useless output: alex@NLDW4-5-20-11:~/ai/ollama$ ./ollama run ll2o >>> Write me a short description of the Adidas brand zekero (16) April 24, 2020 (updated) Adidas is a German multinational sportswear company that was founded in 1948 by Adolf Dassler. The brand is known for its high-quality athletic wear and footwear, including shoes, cleats, and other performance-enhancing gear. Adidas is particularly famous for its iconic three-striped logo and has been associated with some of the most successful athletes in sports history, including Lionel Messi, Kareem Abdul-Jabbar, and Tiger Woods. The brand has a strong focus on innovation and technology, and its products are designed to provide optimal performance and comfort for athletes at all levels. What is this: "zekero (16) April 24, 2020 (updated)"... it's a garbage piece of the output. There is possible some simple solution. If you know how to get rid of this, I would appreciate any advice. Thank you all.
Author
Owner

@mxyng commented on GitHub (Sep 15, 2023):

Based on the output, it looks like you're using the text completion mode as an instruct model. Here's an example of what I mean:

$ ollama run llama2:text
>>> Write me a short description of the Adidas brand
. февраль 23, 2016
The world's leading manufacturer and marketer of sports shoes is the German company adidas AG with headquarters in Germany. The name of the company was formed from the word “Adi" by its founder Adolf Dassler and "das" - meaning "of Adi". The first brand logo has a three-leaf clover, which also appeared on the flag of Bavaria. After World War II, this emblem disappeared from the brand's products. Only in 1972, with the appearance of the first logo adidas, it was revived.
The history of the company begins in 1924 when two brothers Adolf and Rudolf Dassler founded the "Gebrüder Dassler Schuhfabrik" factory, specializing in sports shoes (running tracks). After World War II, a conflict arose between them, which caused the creation of two companies - adidas and Puma. The first name was chosen as a tribute to Adolf's father, the second to Rudolf.
Adidas is one of the largest manufacturers of sports equipment in the world. It produces not only shoes but also clothing (including outerwear), accessories (backpacks, bags and others.), And other gear that can be used for physical activity (helmets, gloves, caps, etc.). The company has a huge number of sports teams (both amateur and professional) around the world.
Adidas is one of the best known brands in the field of sporting goods not only in Germany but throughout the world. The company has been producing high-quality products for more than 90 years, which are sold both directly and through its own network of stores (both online and offline) and franchisees around the world.
In addition to the production of sports shoes, clothes, bags, backpacks, hats and other gear that can be used in sports activities, adidas also offers a line of leisure and casual wear, as well as its own branded collection for children. The company's products are sold through its own network of stores (both online and offline) and franchisees around the world.
The company Adidas is one of the leading manufacturers of sports shoes in the world with a history that began in 1924 when two brothers Adolf and Rudolf Dassler founded the "Gebrüder Dassler Schuhfabrik" factory, specializing in sports shoes (running tracks). After World War II, a conflict arose between them which caused the creation of two companies - adidas and Puma. The first name was chosen as a tribute to Adolf's father, the second to Rudolf.
$ ollama run llama2:chat
>>> Write me a short description of the Adidas brand
 Adidas is a leading global sportswear brand known for its high-quality, stylish, and innovative products. Unterscheidung]Adidas was founded in 1948 by Adolf Dassler, who started producing athletic shoes in the backroom of his mother's laundry business in Herzogenaurach, Germany. Today, Adidas is a global company with a presence in over 100 countries, offering a wide range of products including footwear, apparel, and accessories for men, women, and children.

Adidas is synonymous with quality, performance, and style. The brand is particularly known for its iconic three-striped logo, which represents the union of creativity, functionality, and innovation. Adidas prides itself on creating products that are not only technically advanced but also visually appealing, making it a favorite among athletes and fashion enthusiasts alike.

Adidas has been at the forefront of sportswear technology for decades, introducing groundbreaking designs such as the iconic Stan Smith sneakers and the popular Boost cushioning system. The brand's commitment to innovation has enabled athletes to perform at their best, whether on the football pitch, tennis court, or running track.

Beyond sportswear, Adidas has expanded its offerings to include lifestyle apparel and accessories, such as hoodies, t-shirts, and bags. These products blend functionality with fashion, making it easy for customers to integrate Adidas into their everyday lives.

Adidas's brand values of creativity, diversity, and sustainability are reflected in its collaborations with influential designers and artists, such as Yohji Yamamoto and Kanye West. The brand has also made a commitment to environmental responsibility, using recycled materials and reducing waste in its production processes.

In summary, Adidas is a global sportswear powerhouse renowned for its quality, innovation, and style. From athletic shoes to lifestyle apparel, Adidas offers products that not only perform but also look great, making it a leader in the fashion and sports industries.

The chat fine-tune produces much better results for the type of input you're using and it's the default in ollama.

You might have noticed Unterscheidung] in the output. This seems to be an artifact of the language model when there isn't a system prompt as suggested by Meta.

Here's the result with a simple system prompt:

FROM llama2
SYSTEM "You are a helpful AI assistant."
$ ollama create llama2-with-system
$ ollama run llama2-with-system
>>> /show system
You are a helpful AI assistant.
>>> Write me a short description of the Adidas brand
 Of course, I'd be happy to help! Adidas is a well-known global sportswear brand that was founded in Germany in 1948. The company is known for its high-quality athletic footwear, apparel, and accessories, which are designed to provide comfort, performance, and style for athletes and casual wearers alike.

Adidas is particularly famous for its iconic three-striped logo and has become synonymous with sportswear. The brand has a long history of sponsoring top athletes in various sports such as football, basketball, tennis, and running. Adidas' product line includes shoes, clothing, and accessories for men, women, and children, featuring cutting-edge technology, stylish designs, and innovative materials.

In addition to its sporting heritage, Adidas has also become a fashion powerhouse, with collaborations with high-end designers like Yohji Yamamoto and Stella McCartney, as well as its own in-house design team. The brand is known for its minimalist and modern aesthetic, which appeals to both athletes and fashion enthusiasts.

Adidas has also been at the forefront of sustainability initiatives in the sportswear industry. The company has made significant strides towards reducing its environmental impact through the use of recycled materials, reducing waste, and implementing ethical manufacturing practices.

Overall, Adidas is a brand that embodies both performance and style, making it a popular choice for athletes and fashion enthusiasts alike. Its commitment to sustainability and ethical practices only adds to its appeal as a leader in the sportswear industry.
<!-- gh-comment-id:1721967504 --> @mxyng commented on GitHub (Sep 15, 2023): Based on the output, it looks like you're using the text completion mode as an instruct model. Here's an example of what I mean: ``` $ ollama run llama2:text >>> Write me a short description of the Adidas brand . февраль 23, 2016 The world's leading manufacturer and marketer of sports shoes is the German company adidas AG with headquarters in Germany. The name of the company was formed from the word “Adi" by its founder Adolf Dassler and "das" - meaning "of Adi". The first brand logo has a three-leaf clover, which also appeared on the flag of Bavaria. After World War II, this emblem disappeared from the brand's products. Only in 1972, with the appearance of the first logo adidas, it was revived. The history of the company begins in 1924 when two brothers Adolf and Rudolf Dassler founded the "Gebrüder Dassler Schuhfabrik" factory, specializing in sports shoes (running tracks). After World War II, a conflict arose between them, which caused the creation of two companies - adidas and Puma. The first name was chosen as a tribute to Adolf's father, the second to Rudolf. Adidas is one of the largest manufacturers of sports equipment in the world. It produces not only shoes but also clothing (including outerwear), accessories (backpacks, bags and others.), And other gear that can be used for physical activity (helmets, gloves, caps, etc.). The company has a huge number of sports teams (both amateur and professional) around the world. Adidas is one of the best known brands in the field of sporting goods not only in Germany but throughout the world. The company has been producing high-quality products for more than 90 years, which are sold both directly and through its own network of stores (both online and offline) and franchisees around the world. In addition to the production of sports shoes, clothes, bags, backpacks, hats and other gear that can be used in sports activities, adidas also offers a line of leisure and casual wear, as well as its own branded collection for children. The company's products are sold through its own network of stores (both online and offline) and franchisees around the world. The company Adidas is one of the leading manufacturers of sports shoes in the world with a history that began in 1924 when two brothers Adolf and Rudolf Dassler founded the "Gebrüder Dassler Schuhfabrik" factory, specializing in sports shoes (running tracks). After World War II, a conflict arose between them which caused the creation of two companies - adidas and Puma. The first name was chosen as a tribute to Adolf's father, the second to Rudolf. ``` ``` $ ollama run llama2:chat >>> Write me a short description of the Adidas brand Adidas is a leading global sportswear brand known for its high-quality, stylish, and innovative products. Unterscheidung]Adidas was founded in 1948 by Adolf Dassler, who started producing athletic shoes in the backroom of his mother's laundry business in Herzogenaurach, Germany. Today, Adidas is a global company with a presence in over 100 countries, offering a wide range of products including footwear, apparel, and accessories for men, women, and children. Adidas is synonymous with quality, performance, and style. The brand is particularly known for its iconic three-striped logo, which represents the union of creativity, functionality, and innovation. Adidas prides itself on creating products that are not only technically advanced but also visually appealing, making it a favorite among athletes and fashion enthusiasts alike. Adidas has been at the forefront of sportswear technology for decades, introducing groundbreaking designs such as the iconic Stan Smith sneakers and the popular Boost cushioning system. The brand's commitment to innovation has enabled athletes to perform at their best, whether on the football pitch, tennis court, or running track. Beyond sportswear, Adidas has expanded its offerings to include lifestyle apparel and accessories, such as hoodies, t-shirts, and bags. These products blend functionality with fashion, making it easy for customers to integrate Adidas into their everyday lives. Adidas's brand values of creativity, diversity, and sustainability are reflected in its collaborations with influential designers and artists, such as Yohji Yamamoto and Kanye West. The brand has also made a commitment to environmental responsibility, using recycled materials and reducing waste in its production processes. In summary, Adidas is a global sportswear powerhouse renowned for its quality, innovation, and style. From athletic shoes to lifestyle apparel, Adidas offers products that not only perform but also look great, making it a leader in the fashion and sports industries. ``` The chat fine-tune produces much better results for the type of input you're using and it's the default in ollama. You might have noticed `Unterscheidung]` in the output. This seems to be an artifact of the language model when there isn't a system prompt as suggested by Meta. Here's the result with a simple system prompt: ``` FROM llama2 SYSTEM "You are a helpful AI assistant." ``` ``` $ ollama create llama2-with-system $ ollama run llama2-with-system >>> /show system You are a helpful AI assistant. >>> Write me a short description of the Adidas brand Of course, I'd be happy to help! Adidas is a well-known global sportswear brand that was founded in Germany in 1948. The company is known for its high-quality athletic footwear, apparel, and accessories, which are designed to provide comfort, performance, and style for athletes and casual wearers alike. Adidas is particularly famous for its iconic three-striped logo and has become synonymous with sportswear. The brand has a long history of sponsoring top athletes in various sports such as football, basketball, tennis, and running. Adidas' product line includes shoes, clothing, and accessories for men, women, and children, featuring cutting-edge technology, stylish designs, and innovative materials. In addition to its sporting heritage, Adidas has also become a fashion powerhouse, with collaborations with high-end designers like Yohji Yamamoto and Stella McCartney, as well as its own in-house design team. The brand is known for its minimalist and modern aesthetic, which appeals to both athletes and fashion enthusiasts. Adidas has also been at the forefront of sustainability initiatives in the sportswear industry. The company has made significant strides towards reducing its environmental impact through the use of recycled materials, reducing waste, and implementing ethical manufacturing practices. Overall, Adidas is a brand that embodies both performance and style, making it a popular choice for athletes and fashion enthusiasts alike. Its commitment to sustainability and ethical practices only adds to its appeal as a leader in the sportswear industry. ```
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#26000