github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #6286] Context window size cannot be changed #50449

New Issue
Open
opened 2026-04-28 15:55:11 -05:00 by GiteaMirror · 21 comments
GiteaMirror commented 2026-04-28 15:55:11 -05:00
Owner
Copy Link

Originally created by @mihaelagrigore on GitHub (Aug 9, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/6286

What is the issue?

I see this issue has been partially reported, but none of the previous reports seem to be extensive in their tests of possible methods to set this option.

The problem:
Ollama server truncates the input to 2048 tokens regardless of the chat completion API used.

My setup:
I tried Ollama on my local computer with Windows and CPU only
As well as on a Linux machine, CPU only

I tried several models: llama3, gemma2, mistral

I tried several APIs: ollama's, Langchain, OpenAI

I see Ollama server starts by default on my two machines with a context window size of 8192.
level=INFO source=server.go:617 msg="waiting for server to become available" status="llm server loading model" llama_new_context_with_model: n_ctx = 8192

But just in case, I also tried:
/set parameter num_ctx 8192
Which, as a side note, on Linux machine ends up setting a window size of 4*8192, so I had to restart the server.

The code I use for getting a chat completion:
completion = self.llm_client.chat(model=self.model, messages=[
{"role": "system", "content": self.context},
{"role": "user", "content": question}
],
options=dict(temperature=temperature, n_ctx=n_ctx)
)
This is ignored regardless of which API I use (ollama, langchain or openAI) or on which machine I run (Windows or Linux).
The server logs show:
INFO [update_slots] input truncated | n_ctx=2048 n_erase=3157 n_keep=4 n_left=2044 n_shift=1022 tid="139731362228096" timestamp=1723208612

The only way I can get ollama to use a context window of a given size is by not using an API and just making the call directly through the requests library. But this is much slower than using langchain API (which seems to be the fastest of the 3).

url = base_url + "/api/chat"
model = self.model

            `payload = {
                "model": model,
                "messages": [
                    {"role": "system", "content": self.context},
                    {"role": "user", "content": question}
                ],
                "stream": False,
                "options": {
                    "num_ctx": n_ctx,
                    "temperature": self.temperature,
                    "max_tokens": self.max_tokens
                }
            }
            headers = {
                "Content-Type": "application/json"
            }
            response = requests.post(url, data=json.dumps(payload), headers=headers)
            response.raise_for_status()`

OS

Linux, Windows

GPU

No response

CPU

No response

Ollama version

0.3.4

Originally created by @mihaelagrigore on GitHub (Aug 9, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/6286 ### What is the issue? I see this issue has been partially reported, but none of the previous reports seem to be extensive in their tests of possible methods to set this option. The problem: Ollama server truncates the input to 2048 tokens regardless of the chat completion API used. My setup: I tried Ollama on my local computer with Windows and CPU only As well as on a Linux machine, CPU only I tried several models: llama3, gemma2, mistral I tried several APIs: ollama's, Langchain, OpenAI I see Ollama server starts by default on my two machines with a context window size of 8192. ` level=INFO source=server.go:617 msg="waiting for server to become available" status="llm server loading model" llama_new_context_with_model: n_ctx = 8192 ` But just in case, I also tried: `/set parameter num_ctx 8192` Which, as a side note, on Linux machine ends up setting a window size of 4*8192, so I had to restart the server. The code I use for getting a chat completion: `completion = self.llm_client.chat(model=self.model, messages=[` ` {"role": "system", "content": self.context},` ` {"role": "user", "content": question}` ` ],` ` options=dict(temperature=temperature, n_ctx=n_ctx)` `)` This is ignored regardless of which API I use (ollama, langchain or openAI) or on which machine I run (Windows or Linux). The server logs show: `INFO [update_slots] input truncated | n_ctx=2048 n_erase=3157 n_keep=4 n_left=2044 n_shift=1022 tid="139731362228096" timestamp=1723208612` The only way I can get ollama to use a context window of a given size is by not using an API and just making the call directly through the **requests** library. But this is much slower than using langchain API (which seems to be the fastest of the 3). `url = base_url + "/api/chat"` `model = self.model` `payload = { "model": model, "messages": [ {"role": "system", "content": self.context}, {"role": "user", "content": question} ], "stream": False, "options": { "num_ctx": n_ctx, "temperature": self.temperature, "max_tokens": self.max_tokens } } headers = { "Content-Type": "application/json" } response = requests.post(url, data=json.dumps(payload), headers=headers) response.raise_for_status()` ### OS Linux, Windows ### GPU _No response_ ### CPU _No response_ ### Ollama version 0.3.4
GiteaMirror added the bug label 2026-04-28 15:55:12 -05:00
GiteaMirror commented 2026-04-28 15:55:15 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 9, 2024):

The OpenAI compatibility endpoints don't support setting the context size as that's not part of the official OpenAI API standard. If you want to use the OpenAI endpoints with a different sized context window, you need to create a new model that has a default context window of the desired size: https://github.com/ollama/ollama/issues/5965#issuecomment-2252354726.

The context window ended up as 4*8192 because either you have OLLAMA_NUM_PARALLEL=4 or it's unset, and ollama saw that you had plenty of resources and used a default of 4 instead of 1. You can override this default by setting OLLAMA_NUM_PARALLEL=1 in the server environment.

<!-- gh-comment-id:2278101819 --> @rick-github commented on GitHub (Aug 9, 2024): The OpenAI compatibility endpoints don't support setting the context size as that's not part of the official OpenAI API standard. If you want to use the OpenAI endpoints with a different sized context window, you need to create a new model that has a default context window of the desired size: https://github.com/ollama/ollama/issues/5965#issuecomment-2252354726. The context window ended up as 4*8192 because either you have `OLLAMA_NUM_PARALLEL=4` or it's unset, and ollama saw that you had plenty of resources and used a default of 4 instead of 1. You can override this default by setting `OLLAMA_NUM_PARALLEL=1` in the server environment.
GiteaMirror commented 2026-04-28 15:55:17 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 9, 2024):

n_ctx in your chat completion example should be num_ctx.

<!-- gh-comment-id:2278134165 --> @rick-github commented on GitHub (Aug 9, 2024): `n_ctx` in your chat completion example should be `num_ctx`.
GiteaMirror commented 2026-04-28 15:55:20 -05:00
Author
Owner
Copy Link

@MaxJa4 commented on GitHub (Aug 9, 2024):

The context window ended up as 4*8192 because either you have OLLAMA_NUM_PARALLEL=4 or it's unset, and ollama saw that you had plenty of resources and used a default of 4 instead of 1. You can override this default by setting OLLAMA_NUM_PARALLEL=1 in the server environment.

I was wondering for so long why this is happening... thanks for pointing that out! Totally makes sense if I think about it now.

<!-- gh-comment-id:2278259621 --> @MaxJa4 commented on GitHub (Aug 9, 2024): > The context window ended up as 4*8192 because either you have OLLAMA_NUM_PARALLEL=4 or it's unset, and ollama saw that you had plenty of resources and used a default of 4 instead of 1. You can override this default by setting OLLAMA_NUM_PARALLEL=1 in the server environment. I was wondering for so long why this is happening... thanks for pointing that out! Totally makes sense if I think about it now.
GiteaMirror commented 2026-04-28 15:55:21 -05:00
Author
Owner
Copy Link

@MeinDeutschkurs commented on GitHub (Aug 23, 2024):

def generate_text(prompt, model='mistral-nemo', options=dict(num_ctx=128000)):
    response = ollama.generate(model=model, prompt=prompt, options=options)
    return response['response']

What do I wrong? After about two terminal pages, the output is cut off. It seems to be that it is using just between 1000 and 1100 tokens (count by mistral Nemo token counter) for output. Even when I output it to a file.

<!-- gh-comment-id:2306671917 --> @MeinDeutschkurs commented on GitHub (Aug 23, 2024): ``` def generate_text(prompt, model='mistral-nemo', options=dict(num_ctx=128000)): response = ollama.generate(model=model, prompt=prompt, options=options) return response['response'] ``` What do I wrong? After about two terminal pages, the output is cut off. It seems to be that it is using just between 1000 and 1100 tokens (count by mistral Nemo token counter) for output. Even when I output it to a file.
GiteaMirror commented 2026-04-28 15:55:23 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 23, 2024):

What prompt?

<!-- gh-comment-id:2306867791 --> @rick-github commented on GitHub (Aug 23, 2024): What prompt?
GiteaMirror commented 2026-04-28 15:55:24 -05:00
Author
Owner
Copy Link

@MeinDeutschkurs commented on GitHub (Aug 23, 2024):

I cannot quote it, but any prompt that should produce more text. You can test this by the use of an auto refiner. E.g:

Pseudo-Code:

  1. Write a text about "any topic". return: current_iteration and print it

  2. Loop (4)

  • <prompt>{initial_prompt}</prompt> <text>{current_iteration}</text> Suggest, how we could make the text better and/or how we can enhance it. return: suggestions
  • <prompt>{initial_prompt}</prompt><original text>{current_iteration}</original_text> <suggestion>{suggestions}</suggestions> Please use the suggestions to write a better version of original_text. return: as new current_iteration and print it
  1. output

Most of the iterations are cut off, even 128000 num_ctx.

I use macOS, M2 and 192GB ram. (About 140 GB as VRAM)

<!-- gh-comment-id:2307067911 --> @MeinDeutschkurs commented on GitHub (Aug 23, 2024): I cannot quote it, but any prompt that should produce more text. You can test this by the use of an auto refiner. E.g: Pseudo-Code: 1) Write a text about "any topic". return: current_iteration and print it 2) Loop (4) - &lt;prompt>{initial_prompt}&lt;/prompt> &lt;text>{current_iteration}&lt;/text> Suggest, how we could make the text better and/or how we can enhance it. return: suggestions - &lt;prompt>{initial_prompt}&lt;/prompt>&lt;original text>{current_iteration}&lt;/original_text> &lt;suggestion>{suggestions}&lt;/suggestions> Please use the suggestions to write a better version of original_text. return: as new current_iteration and print it 3) output Most of the iterations are cut off, even 128000 num_ctx. I use macOS, M2 and 192GB ram. (About 140 GB as VRAM)
GiteaMirror commented 2026-04-28 15:55:25 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 23, 2024):

What's the goal? Can you provide a complete python script that demonstrates the problem?

<!-- gh-comment-id:2307101565 --> @rick-github commented on GitHub (Aug 23, 2024): What's the goal? Can you provide a complete python script that demonstrates the problem?
GiteaMirror commented 2026-04-28 15:55:25 -05:00
Author
Owner
Copy Link

@MeinDeutschkurs commented on GitHub (Aug 23, 2024):

import argparse
import ollama

def generate_text(prompt, model='mistral-nemo', options=dict(num_ctx=128000)):
    response = ollama.generate(model=model, prompt=prompt, options=options)
    return response['response']

def main():
    parser = argparse.ArgumentParser(description="Text enhancement script")
    parser.add_argument("--prompt", type=str, required=True, help="Initial prompt")
    parser.add_argument("--loops", type=int, default=4, help="Number of enhancement loops")
    args = parser.parse_args()

    initial_prompt = args.prompt
    current_iteration = generate_text(initial_prompt)
    print(f"\n\nInitial text (Iteration 0):\n{current_iteration}\n")

    for i in range(args.loops):
        # Generate suggestions
        suggestion_prompt = f"<prompt>{initial_prompt}</prompt> <text>{current_iteration}</text> Suggest how we could make the text better and/or how we can enhance it."
        suggestions = generate_text(suggestion_prompt)
        print(f"\n\nSuggestions (Iteration {i+1}):\n{suggestions}\n")

        # Generate improved text
        improvement_prompt = f"<prompt>{initial_prompt}</prompt><original_text>{current_iteration}</original_text> <suggestion>{suggestions}</suggestion> Please use the suggestions to write a better version of original_text. NO GRAPHICS/LINKS/DIAGRAMS!!"
        current_iteration = generate_text(improvement_prompt)
        print(f"\n\nImproved text (Iteration {i+1}):\n{current_iteration}\n")

if __name__ == "__main__":
    main()

The goal is to improve the quality and richness of the text.

python quicktext.py --prompt """Write an article about 'Living in Las Vegas'""" --loop 5

Just have a look onto the outputs. You will see that these are cut off after some iterations.

<!-- gh-comment-id:2307109565 --> @MeinDeutschkurs commented on GitHub (Aug 23, 2024): ``` import argparse import ollama def generate_text(prompt, model='mistral-nemo', options=dict(num_ctx=128000)): response = ollama.generate(model=model, prompt=prompt, options=options) return response['response'] def main(): parser = argparse.ArgumentParser(description="Text enhancement script") parser.add_argument("--prompt", type=str, required=True, help="Initial prompt") parser.add_argument("--loops", type=int, default=4, help="Number of enhancement loops") args = parser.parse_args() initial_prompt = args.prompt current_iteration = generate_text(initial_prompt) print(f"\n\nInitial text (Iteration 0):\n{current_iteration}\n") for i in range(args.loops): # Generate suggestions suggestion_prompt = f"<prompt>{initial_prompt}</prompt> <text>{current_iteration}</text> Suggest how we could make the text better and/or how we can enhance it." suggestions = generate_text(suggestion_prompt) print(f"\n\nSuggestions (Iteration {i+1}):\n{suggestions}\n") # Generate improved text improvement_prompt = f"<prompt>{initial_prompt}</prompt><original_text>{current_iteration}</original_text> <suggestion>{suggestions}</suggestion> Please use the suggestions to write a better version of original_text. NO GRAPHICS/LINKS/DIAGRAMS!!" current_iteration = generate_text(improvement_prompt) print(f"\n\nImproved text (Iteration {i+1}):\n{current_iteration}\n") if __name__ == "__main__": main() ``` The goal is to improve the quality and richness of the text. `python quicktext.py --prompt """Write an article about 'Living in Las Vegas'""" --loop 5` Just have a look onto the outputs. You will see that these are cut off after some iterations.
GiteaMirror commented 2026-04-28 15:55:26 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 23, 2024):

I ran the script with the suggested prompt, final output below. The last line (By incorporating these enhancements and suggestions, this revised article...) seems to be an appropriate conclusion implying there's no cutoff. If this is not what you are finding, then server logs generated with OLLAMA_DEBUG=1 may provide some insight.

Improved text (Iteration 5):
**Title: Living in Las Vegas: The Complete Guide Beyond the Glitter**

Nestled between desert mountains and expansive valleys, Las Vegas, Nevada's vibrant metropolis, offers more than just its famous Strip and sparkling casinos. With a population of over 650,000 residents, living in Las Vegas presents an affordable housing market, diverse lifestyle options, stunning natural surroundings, and an entertainment scene that caters to every taste. Let's explore the true essence of life beyond Sin City's glittering facade with this comprehensive guide.

**1. Housing Options**

Las Vegas provides various housing choices tailored to different budgets and lifestyles:

* **Downtown Las Vegas**: This historic district offers a mix of vintage buildings converted into loft-style apartments, condos, and single-family homes. Close to the Fremont Street Experience but maintaining a peaceful atmosphere, it's perfect for young professionals and creatives.
        + Personal Perspective: *"I bought my downtown condo for less than $200,000," says John, a graphic designer who's lived there for three years. "I enjoy being within walking distance of the action but still have a quiet retreat at home."*

* **Suburbs**: Las Vegas' suburbs provide family-friendly communities with excellent schools, plenty of parks, and easy access to outdoor recreation:
        + **Summerlin**: Known for its luxury homes, high-end amenities, and prestigious schools.
        + **Henderson** and **Green Valley**: Offer affordable housing options with a relaxed suburban feel.

        Quoted Price Point:
        - Median home prices in Summerlin start around $400,000.
        - Henderson and Green Valley homes average about $350,000.

* **Gated Communities**: Las Vegas has numerous gated communities offering luxury living with enhanced security and privacy:
        + Spanish Trail: An upscale neighborhood featuring lush landscaping and elegant homes.
        + MacDonald Ranch: A secluded community with a country club atmosphere.

**2. Cost of Living**

One significant advantage of living in Las Vegas is its relatively low cost of living compared to other major U.S. cities:

|  | Las Vegas | Los Angeles | Chicago |
|---|---|---|---|
| **Cost of Living Index (NerdWallet)** | 86.4 | 153.2 | 97.0 |
| **Median Home Price** | $350,000 | $800,000+ | $280,000 |
| **Average Summer Temperature** | 95°F (35°C) | 84°F (29°C) | 86°F (30°C) |

* Housing costs are notably lower in Las Vegas.
        + Median home prices around $350,000.
        + Median monthly rent for a one-bedroom apartment at approximately $900.

**3. Entertainment and Culture**

Beyond its world-famous shows and casinos, Las Vegas boasts a rich cultural scene with something for everyone:

* **Cultural Institutions**:
        + The Smith Center for the Performing Arts: Hosts Broadway shows, concerts, and other performances.
        + The Neon Museum: Preserves iconic Las Vegas signs in its Boneyard.
        + The Mob Museum: Delves into the city's organized crime history.

* **Outdoor Activities**: Las Vegas' unique desert location offers ample opportunities for outdoor pursuits:
        + Red Rock Canyon National Conservation Area: Provides hiking, camping, and stunning scenery just minutes from the Strip.
        + Valley of Fire State Park: Showcases breathtaking red rock formations about an hour's drive northeast of the city.
        + Spring Mountain Ranch State Park: Offers a historic ranch house, beautiful gardens, and picnic areas.

* **Sports**: Las Vegas has become a major sports hub with professional teams like the NFL's Raiders, the NHL's Golden Knights, and the WNBA's Aces:
        + Catch games at state-of-the-art stadiums and arenas throughout the valley.

* **Arts and Creativity**:
        + Downtown Arts District: Home to various art galleries, murals, and monthly art walks.
        + 18b Arts District: A vibrant hub for local artists, creative businesses, and live music venues.

**4. Food Scene**

Las Vegas' diverse food scene caters to every taste bud:

* Celebrity chef-owned restaurants and authentic ethnic cuisine provide a wide variety of dining options.
* Popular dishes include steakhouse fare, fusion cuisine, Mexican-inspired flavors, and plant-based offerings.
* Food halls like Downtown Container Park and The District at Green Valley Ranch house numerous eateries under one roof.

**5. Weather**

Las Vegas enjoys a desert climate with hot summers and mild winters:

| Season | Average Temperature (F) |
|---|---|
| Summer (June-September) | 70°F to 104°F (21°C to 40°C) |
| Winter (December-March) | 38°F to 65°F (3°C to 18°C) |

* Las Vegas receives less than four inches of rain per year, making it an ideal location for those who prefer dry heat and plenty of sunshine.

**6. Education**

Las Vegas offers various public and private school options:

* **Public Schools**: Clark County School District operates the largest public school system in Nevada, with Magnet Schools and specialized programs available.
* **Private Schools**:
        + Faith Lutheran Academy
        + The Meadows School
        + Adelson Educational Campus

* **Higher Education**:
        + University of Nevada, Las Vegas (UNLV)
        + Nevada State College

**7. Healthcare**

Las Vegas boasts several prominent healthcare providers:

* University Medical Center: A Level I trauma center and teaching hospital serving as the primary provider for Las Vegas' most critically ill and injured patients.
* Dignity Health - St. Rose Dominican Hospitals: Offers multiple campuses throughout the valley, providing comprehensive care for residents in various neighborhoods.
* Kidney Care Institute: Provides advanced renal care services at several locations across the city.

**8. Community**

Despite its transient reputation, Las Vegas has a close-knit community with numerous opportunities to connect:

* Local events like Life is Beautiful and the Las Vegas Greek Food Festival bring people together each year.
* Neighborhood associations and social clubs help residents build lasting connections through shared interests.
* Community gatherings at parks and recreation centers foster a sense of belonging among Las Vegas residents.

**9. Hidden Gems**

Beyond its famous attractions, Las Vegas has its share of lesser-known spots waiting to be discovered:

* Fifth Street School: An historic arts center offering classes and events in downtown Las Vegas.
* Emergency Airstrip: A remnant of WWII now serving as a public park with beautiful views overlooking the city.
* Eureka! Las Vegas: A unique casino with a steampunk-themed design and a focus on experiential gaming.

**10. Pros vs. Cons**

Weighing the benefits and challenges of living in Las Vegas:

| Pros | Cons |
|---|---|
| Affordable housing | Limited public transportation |
| Vibrant entertainment scene | Traffic congestion, especially during peak hours |
| Desert climate with plenty of sunshine | Transient population may impact community stability |
| Close proximity to outdoor recreation | Air quality can be poor due to desert dust and inversions |

**11. Relocation Tips**

Consider the following tips if you're thinking about relocating to Las Vegas:

* Visit the city during different times of year to experience its varying climates.
* Familiarize yourself with neighborhoods before searching for housing or jobs.
* Network with local professionals through LinkedIn groups or online forums like Reddit's r/LasVegas subreddit.

**12. Seasonal Living Considerations**

Living in Las Vegas differs between seasons:

* **Summer (June-September)**:
        + Temperatures soar above 100°F (38°C) on some days.
        + Tourist crowds swell, leading to longer lines and increased noise at popular attractions.
        + Local events like Electric Daisy Carnival and the Neon Museum's monthly Boneyard Tours add excitement during these months.

* **Winter (December-March)**:
        + Mild winters allow for outdoor activities year-round.
        + Festive celebrations, such as Christmas shows and New Year's Eve fireworks displays, bring added cheer.
        + The Strip transforms into a winter wonderland with elaborate holiday decorations.

**13. Historical Context**

Las Vegas' evolution from a small desert railroad stop to a global entertainment hub is fascinating:

* In the 1930s, organized crime syndicates began shaping Las Vegas into America's premier gambling destination.
* Following World War II, the city grew rapidly as it became known for its high-stakes casinos and vibrant nightlife.
* Today, Las Vegas continues to reinvent itself, embracing family-friendly attractions, conventions, and upscale dining options alongside its iconic gaming scene.

**14. Diversity and Inclusion**

Las Vegas actively promotes diversity and inclusion:

* Annual events like Las Vegas Pride celebrate the city's LGBTQ+ community.
* Cultural festivals showcase various ethnic heritages.
* Local initiatives focus on affordable housing and community development to ensure Las Vegas remains welcoming for all residents.

**15. Pet-Friendly Living**

Las Vegas is generally pet-friendly with numerous dog parks and pet-friendly amenities throughout neighborhoods:

* Popular dog parks include Sunset Park (near downtown) and Centennial Hills Dog Park (in the northwest valley).
* Many apartments, condominiums, and restaurants offer pet-friendly spaces.
* Regulations require pets to be licensed, vaccinated against rabies, and kept on a leash when in public areas.

**16. Expert Insights**

Incorporating interviews with local experts provides unique perspectives on living in Las Vegas:

* Real Estate Agent Perspective:
        + *"Las Vegas offers an affordable housing market compared to other major U.S. cities," says Sarah, a veteran real estate agent.*

* Educator Perspective:
        + *"Clark County School District provides numerous Magnet Schools and specialized programs to accommodate diverse learning needs," notes Mr. Johnson, a middle school principal.*

**17. FAQs**

* **What are some popular neighborhoods for families in Las Vegas?**
        + Summerlin, Henderson, Green Valley, Centennial Hills, and the northwest valley offer excellent family-friendly communities with good schools and plenty of amenities.

* **How is the job market in Las Vegas?**
        + Las Vegas has a diverse job market, with significant opportunities in hospitality, gaming, entertainment, construction, healthcare, education, and other industries. The unemployment rate tends to be slightly higher than the national average due to the city's reliance on tourism, but the economy remains robust overall.

* **Is public transportation available in Las Vegas?**
        + Yes, Las Vegas has a public bus system called Citizens Area Transit (C.A.T.) operated by the Regional Transportation Commission of Southern Nevada. The city also offers the Downtown Loop, an electric bus service that connects downtown attractions and the Fremont Street Experience. However, due to the sprawling nature of the city, owning a car is often necessary for easy navigation.

* **What are some popular outdoor activities in Las Vegas?**
        + Hiking at Red Rock Canyon National Conservation Area, exploring Valley of Fire State Park, visiting Hoover Dam, playing golf at one of many courses throughout the valley, and enjoying picnics or sports at local parks like Sunset Park and Wetlands Park.

By incorporating these enhancements and suggestions, this revised article provides a comprehensive and engaging look into what it's truly like to live in Las Vegas beyond the glitter. Whether you're considering relocating or simply curious about life behind the scenes, there's something for everyone in Sin City.
<!-- gh-comment-id:2307292747 --> @rick-github commented on GitHub (Aug 23, 2024): I ran the script with the suggested prompt, final output below. The last line (`By incorporating these enhancements and suggestions, this revised article...`) seems to be an appropriate conclusion implying there's no cutoff. If this is not what you are finding, then server logs generated with `OLLAMA_DEBUG=1` may provide some insight. ``` Improved text (Iteration 5): **Title: Living in Las Vegas: The Complete Guide Beyond the Glitter** Nestled between desert mountains and expansive valleys, Las Vegas, Nevada's vibrant metropolis, offers more than just its famous Strip and sparkling casinos. With a population of over 650,000 residents, living in Las Vegas presents an affordable housing market, diverse lifestyle options, stunning natural surroundings, and an entertainment scene that caters to every taste. Let's explore the true essence of life beyond Sin City's glittering facade with this comprehensive guide. **1. Housing Options** Las Vegas provides various housing choices tailored to different budgets and lifestyles: * **Downtown Las Vegas**: This historic district offers a mix of vintage buildings converted into loft-style apartments, condos, and single-family homes. Close to the Fremont Street Experience but maintaining a peaceful atmosphere, it's perfect for young professionals and creatives. + Personal Perspective: *"I bought my downtown condo for less than $200,000," says John, a graphic designer who's lived there for three years. "I enjoy being within walking distance of the action but still have a quiet retreat at home."* * **Suburbs**: Las Vegas' suburbs provide family-friendly communities with excellent schools, plenty of parks, and easy access to outdoor recreation: + **Summerlin**: Known for its luxury homes, high-end amenities, and prestigious schools. + **Henderson** and **Green Valley**: Offer affordable housing options with a relaxed suburban feel. Quoted Price Point: - Median home prices in Summerlin start around $400,000. - Henderson and Green Valley homes average about $350,000. * **Gated Communities**: Las Vegas has numerous gated communities offering luxury living with enhanced security and privacy: + Spanish Trail: An upscale neighborhood featuring lush landscaping and elegant homes. + MacDonald Ranch: A secluded community with a country club atmosphere. **2. Cost of Living** One significant advantage of living in Las Vegas is its relatively low cost of living compared to other major U.S. cities: | | Las Vegas | Los Angeles | Chicago | |---|---|---|---| | **Cost of Living Index (NerdWallet)** | 86.4 | 153.2 | 97.0 | | **Median Home Price** | $350,000 | $800,000+ | $280,000 | | **Average Summer Temperature** | 95°F (35°C) | 84°F (29°C) | 86°F (30°C) | * Housing costs are notably lower in Las Vegas. + Median home prices around $350,000. + Median monthly rent for a one-bedroom apartment at approximately $900. **3. Entertainment and Culture** Beyond its world-famous shows and casinos, Las Vegas boasts a rich cultural scene with something for everyone: * **Cultural Institutions**: + The Smith Center for the Performing Arts: Hosts Broadway shows, concerts, and other performances. + The Neon Museum: Preserves iconic Las Vegas signs in its Boneyard. + The Mob Museum: Delves into the city's organized crime history. * **Outdoor Activities**: Las Vegas' unique desert location offers ample opportunities for outdoor pursuits: + Red Rock Canyon National Conservation Area: Provides hiking, camping, and stunning scenery just minutes from the Strip. + Valley of Fire State Park: Showcases breathtaking red rock formations about an hour's drive northeast of the city. + Spring Mountain Ranch State Park: Offers a historic ranch house, beautiful gardens, and picnic areas. * **Sports**: Las Vegas has become a major sports hub with professional teams like the NFL's Raiders, the NHL's Golden Knights, and the WNBA's Aces: + Catch games at state-of-the-art stadiums and arenas throughout the valley. * **Arts and Creativity**: + Downtown Arts District: Home to various art galleries, murals, and monthly art walks. + 18b Arts District: A vibrant hub for local artists, creative businesses, and live music venues. **4. Food Scene** Las Vegas' diverse food scene caters to every taste bud: * Celebrity chef-owned restaurants and authentic ethnic cuisine provide a wide variety of dining options. * Popular dishes include steakhouse fare, fusion cuisine, Mexican-inspired flavors, and plant-based offerings. * Food halls like Downtown Container Park and The District at Green Valley Ranch house numerous eateries under one roof. **5. Weather** Las Vegas enjoys a desert climate with hot summers and mild winters: | Season | Average Temperature (F) | |---|---| | Summer (June-September) | 70°F to 104°F (21°C to 40°C) | | Winter (December-March) | 38°F to 65°F (3°C to 18°C) | * Las Vegas receives less than four inches of rain per year, making it an ideal location for those who prefer dry heat and plenty of sunshine. **6. Education** Las Vegas offers various public and private school options: * **Public Schools**: Clark County School District operates the largest public school system in Nevada, with Magnet Schools and specialized programs available. * **Private Schools**: + Faith Lutheran Academy + The Meadows School + Adelson Educational Campus * **Higher Education**: + University of Nevada, Las Vegas (UNLV) + Nevada State College **7. Healthcare** Las Vegas boasts several prominent healthcare providers: * University Medical Center: A Level I trauma center and teaching hospital serving as the primary provider for Las Vegas' most critically ill and injured patients. * Dignity Health - St. Rose Dominican Hospitals: Offers multiple campuses throughout the valley, providing comprehensive care for residents in various neighborhoods. * Kidney Care Institute: Provides advanced renal care services at several locations across the city. **8. Community** Despite its transient reputation, Las Vegas has a close-knit community with numerous opportunities to connect: * Local events like Life is Beautiful and the Las Vegas Greek Food Festival bring people together each year. * Neighborhood associations and social clubs help residents build lasting connections through shared interests. * Community gatherings at parks and recreation centers foster a sense of belonging among Las Vegas residents. **9. Hidden Gems** Beyond its famous attractions, Las Vegas has its share of lesser-known spots waiting to be discovered: * Fifth Street School: An historic arts center offering classes and events in downtown Las Vegas. * Emergency Airstrip: A remnant of WWII now serving as a public park with beautiful views overlooking the city. * Eureka! Las Vegas: A unique casino with a steampunk-themed design and a focus on experiential gaming. **10. Pros vs. Cons** Weighing the benefits and challenges of living in Las Vegas: | Pros | Cons | |---|---| | Affordable housing | Limited public transportation | | Vibrant entertainment scene | Traffic congestion, especially during peak hours | | Desert climate with plenty of sunshine | Transient population may impact community stability | | Close proximity to outdoor recreation | Air quality can be poor due to desert dust and inversions | **11. Relocation Tips** Consider the following tips if you're thinking about relocating to Las Vegas: * Visit the city during different times of year to experience its varying climates. * Familiarize yourself with neighborhoods before searching for housing or jobs. * Network with local professionals through LinkedIn groups or online forums like Reddit's r/LasVegas subreddit. **12. Seasonal Living Considerations** Living in Las Vegas differs between seasons: * **Summer (June-September)**: + Temperatures soar above 100°F (38°C) on some days. + Tourist crowds swell, leading to longer lines and increased noise at popular attractions. + Local events like Electric Daisy Carnival and the Neon Museum's monthly Boneyard Tours add excitement during these months. * **Winter (December-March)**: + Mild winters allow for outdoor activities year-round. + Festive celebrations, such as Christmas shows and New Year's Eve fireworks displays, bring added cheer. + The Strip transforms into a winter wonderland with elaborate holiday decorations. **13. Historical Context** Las Vegas' evolution from a small desert railroad stop to a global entertainment hub is fascinating: * In the 1930s, organized crime syndicates began shaping Las Vegas into America's premier gambling destination. * Following World War II, the city grew rapidly as it became known for its high-stakes casinos and vibrant nightlife. * Today, Las Vegas continues to reinvent itself, embracing family-friendly attractions, conventions, and upscale dining options alongside its iconic gaming scene. **14. Diversity and Inclusion** Las Vegas actively promotes diversity and inclusion: * Annual events like Las Vegas Pride celebrate the city's LGBTQ+ community. * Cultural festivals showcase various ethnic heritages. * Local initiatives focus on affordable housing and community development to ensure Las Vegas remains welcoming for all residents. **15. Pet-Friendly Living** Las Vegas is generally pet-friendly with numerous dog parks and pet-friendly amenities throughout neighborhoods: * Popular dog parks include Sunset Park (near downtown) and Centennial Hills Dog Park (in the northwest valley). * Many apartments, condominiums, and restaurants offer pet-friendly spaces. * Regulations require pets to be licensed, vaccinated against rabies, and kept on a leash when in public areas. **16. Expert Insights** Incorporating interviews with local experts provides unique perspectives on living in Las Vegas: * Real Estate Agent Perspective: + *"Las Vegas offers an affordable housing market compared to other major U.S. cities," says Sarah, a veteran real estate agent.* * Educator Perspective: + *"Clark County School District provides numerous Magnet Schools and specialized programs to accommodate diverse learning needs," notes Mr. Johnson, a middle school principal.* **17. FAQs** * **What are some popular neighborhoods for families in Las Vegas?** + Summerlin, Henderson, Green Valley, Centennial Hills, and the northwest valley offer excellent family-friendly communities with good schools and plenty of amenities. * **How is the job market in Las Vegas?** + Las Vegas has a diverse job market, with significant opportunities in hospitality, gaming, entertainment, construction, healthcare, education, and other industries. The unemployment rate tends to be slightly higher than the national average due to the city's reliance on tourism, but the economy remains robust overall. * **Is public transportation available in Las Vegas?** + Yes, Las Vegas has a public bus system called Citizens Area Transit (C.A.T.) operated by the Regional Transportation Commission of Southern Nevada. The city also offers the Downtown Loop, an electric bus service that connects downtown attractions and the Fremont Street Experience. However, due to the sprawling nature of the city, owning a car is often necessary for easy navigation. * **What are some popular outdoor activities in Las Vegas?** + Hiking at Red Rock Canyon National Conservation Area, exploring Valley of Fire State Park, visiting Hoover Dam, playing golf at one of many courses throughout the valley, and enjoying picnics or sports at local parks like Sunset Park and Wetlands Park. By incorporating these enhancements and suggestions, this revised article provides a comprehensive and engaging look into what it's truly like to live in Las Vegas beyond the glitter. Whether you're considering relocating or simply curious about life behind the scenes, there's something for everyone in Sin City. ```
GiteaMirror commented 2026-04-28 15:55:28 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 23, 2024):

I re-ran it with num_ctx=2048, output still seems reasonable:

Improved text (Iteration 5):
**Welcome Home: A Comprehensive Guide to Living in Las Vegas**

As you sip your morning coffee, the gentle crowing of your neighbor's rooster floats through your open window. Welcome to life beyond the Strip, where Las Vegas residents enjoy a unique blend of urban excitement and suburban tranquility. Let's explore what it's truly like to call this entertainment capital home.

**Demographics & Cost of Living**

*Average Age:* 37.8 years (U.S. average: 38.5)
*Average Income:* $46,000 (U.S. average: $51,200)
*Cost of Living Index:* 98.2 (U.S. average: 100)

**Housing**

Las Vegas offers a diverse range of housing options to suit various preferences and budgets:

- **Apartments:** Studios to three-bedrooms, ranging from $700 to $2,500 per month.
- **Townhouses/Condos:** Starting at around $150,000, with prices varying depending on size and location.
- **Single-family homes:** Prices range widely, from approximately $150,000 to over $1 million.

**Neighborhood Spotlights**

*Summerlin:* A master-planned community known for its family-friendly atmosphere, abundant parks, and top-rated schools. The average home price is around $350,000, with a mix of single-family homes, townhouses, and apartments.

*Henderson:* Located just southeast of Las Vegas, Henderson offers a slower pace of life while still providing easy access to the Strip. Residents enjoy a variety of amenities, including parks, recreation centers, and numerous shopping and dining options. The average home price is approximately $280,000.

*Downtown Las Vegas:* Experience vibrant city living in the heart of it all. With an average home price around $350,000, you'll be steps away from world-class entertainment, dining, and arts.

**Culture & Lifestyle**

Life beyond the Strip is filled with culture, festivals, and year-round outdoor activities:

- **Arts & Entertainment:** Explore The Smith Center for the Performing Arts, Neon Museum, or catch an indie film at the historic Neon Movies.
- **Festivals:** From the Life is Beautiful street art festival to the Electric Daisy Carnival music event, there's always something happening.
- **Outdoor Activities:** Hike nearby mountains like Red Rock Canyon or explore the Mojave Desert. Plus, enjoy plenty of golf courses and swimming pools.

**Working & Commuting**

Las Vegas' job market is diverse, with significant industries in hospitality, gaming, and technology:

- **Employment:** Top employers include Caesars Entertainment, Wynn Resorts, and the Las Vegas Sands Corp.
- **Commuting:** Public transportation includes the RTC bus system and the Las Vegas Monorail. Car ownership is also common due to sprawling suburbs.

**Challenges & Drawbacks**

While there's plenty to love about Sin City, be aware of these potential challenges:

- **Transient Employment:** Many jobs are tied to tourism, leading to temporary positions.
- **Summer Heat:** Las Vegas' intense summer heat means higher energy bills for air conditioning.
- **Noise Pollution:** Living near the Strip or popular nightlife areas can mean dealing with noise at all hours.

**Personal Perspectives**

*Meet the Johnsons:*
"After years of visiting, we decided to make Las Vegas our permanent home. We love that our kids can grow up with plenty of parks and a short drive from nature, yet still enjoy world-class entertainment downtown." - Sarah Johnson, Summerlin resident & mom of two

*Meet Carlos:*
"Austin-born Carlos moved here for work and fell in love with the city's diversity and energy. I live in Downtown Las Vegas because it's central to everything, and I love having all that vibrancy just steps away." - Carlos Martinez, Downtown Las Vegas resident & marketing professional

**Practical Information**

- **Relocating:** Obtain a Nevada ID at your local DMV office; register your vehicle with the Nevada Department of Motor Vehicles.
- **Healthcare:** Check out options like Mountain's View Hospital Medical Center or Valley Hospital Medical Center.
- **Education:** Las Vegas is home to several top-rated schools and universities, including UNLV.

**Life Beyond the Strip**

Las Vegas is more than just the glitz and glamour of the Strip. With diverse neighborhoods, a thriving cultural scene, and ample outdoor activities, it's easy to see why so many people choose to call Sin City home.

Whether you're seeking urban excitement or suburban tranquility, there's a place for you in Las Vegas. So, come on in – and remember, what happens here might just become your new normal.
<!-- gh-comment-id:2307357613 --> @rick-github commented on GitHub (Aug 23, 2024): I re-ran it with num_ctx=2048, output still seems reasonable: ``` Improved text (Iteration 5): **Welcome Home: A Comprehensive Guide to Living in Las Vegas** As you sip your morning coffee, the gentle crowing of your neighbor's rooster floats through your open window. Welcome to life beyond the Strip, where Las Vegas residents enjoy a unique blend of urban excitement and suburban tranquility. Let's explore what it's truly like to call this entertainment capital home. **Demographics & Cost of Living** *Average Age:* 37.8 years (U.S. average: 38.5) *Average Income:* $46,000 (U.S. average: $51,200) *Cost of Living Index:* 98.2 (U.S. average: 100) **Housing** Las Vegas offers a diverse range of housing options to suit various preferences and budgets: - **Apartments:** Studios to three-bedrooms, ranging from $700 to $2,500 per month. - **Townhouses/Condos:** Starting at around $150,000, with prices varying depending on size and location. - **Single-family homes:** Prices range widely, from approximately $150,000 to over $1 million. **Neighborhood Spotlights** *Summerlin:* A master-planned community known for its family-friendly atmosphere, abundant parks, and top-rated schools. The average home price is around $350,000, with a mix of single-family homes, townhouses, and apartments. *Henderson:* Located just southeast of Las Vegas, Henderson offers a slower pace of life while still providing easy access to the Strip. Residents enjoy a variety of amenities, including parks, recreation centers, and numerous shopping and dining options. The average home price is approximately $280,000. *Downtown Las Vegas:* Experience vibrant city living in the heart of it all. With an average home price around $350,000, you'll be steps away from world-class entertainment, dining, and arts. **Culture & Lifestyle** Life beyond the Strip is filled with culture, festivals, and year-round outdoor activities: - **Arts & Entertainment:** Explore The Smith Center for the Performing Arts, Neon Museum, or catch an indie film at the historic Neon Movies. - **Festivals:** From the Life is Beautiful street art festival to the Electric Daisy Carnival music event, there's always something happening. - **Outdoor Activities:** Hike nearby mountains like Red Rock Canyon or explore the Mojave Desert. Plus, enjoy plenty of golf courses and swimming pools. **Working & Commuting** Las Vegas' job market is diverse, with significant industries in hospitality, gaming, and technology: - **Employment:** Top employers include Caesars Entertainment, Wynn Resorts, and the Las Vegas Sands Corp. - **Commuting:** Public transportation includes the RTC bus system and the Las Vegas Monorail. Car ownership is also common due to sprawling suburbs. **Challenges & Drawbacks** While there's plenty to love about Sin City, be aware of these potential challenges: - **Transient Employment:** Many jobs are tied to tourism, leading to temporary positions. - **Summer Heat:** Las Vegas' intense summer heat means higher energy bills for air conditioning. - **Noise Pollution:** Living near the Strip or popular nightlife areas can mean dealing with noise at all hours. **Personal Perspectives** *Meet the Johnsons:* "After years of visiting, we decided to make Las Vegas our permanent home. We love that our kids can grow up with plenty of parks and a short drive from nature, yet still enjoy world-class entertainment downtown." - Sarah Johnson, Summerlin resident & mom of two *Meet Carlos:* "Austin-born Carlos moved here for work and fell in love with the city's diversity and energy. I live in Downtown Las Vegas because it's central to everything, and I love having all that vibrancy just steps away." - Carlos Martinez, Downtown Las Vegas resident & marketing professional **Practical Information** - **Relocating:** Obtain a Nevada ID at your local DMV office; register your vehicle with the Nevada Department of Motor Vehicles. - **Healthcare:** Check out options like Mountain's View Hospital Medical Center or Valley Hospital Medical Center. - **Education:** Las Vegas is home to several top-rated schools and universities, including UNLV. **Life Beyond the Strip** Las Vegas is more than just the glitz and glamour of the Strip. With diverse neighborhoods, a thriving cultural scene, and ample outdoor activities, it's easy to see why so many people choose to call Sin City home. Whether you're seeking urban excitement or suburban tranquility, there's a place for you in Las Vegas. So, come on in – and remember, what happens here might just become your new normal. ```
GiteaMirror commented 2026-04-28 15:55:29 -05:00
Author
Owner
Copy Link

@MeinDeutschkurs commented on GitHub (Aug 24, 2024):

Strange. And what about num_ctx 128000? I'm interested in longer text, not in num_ctx <4000 ... As said, num_ctx does not work, because anything does not work properly.

And what about iterations 1 to 4?

<!-- gh-comment-id:2307957005 --> @MeinDeutschkurs commented on GitHub (Aug 24, 2024): Strange. And what about num_ctx 128000? I'm interested in longer text, not in num_ctx <4000 ... As said, num_ctx does not work, because anything does not work properly. And what about iterations 1 to 4?
GiteaMirror commented 2026-04-28 15:55:30 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 24, 2024):

The first run (https://github.com/ollama/ollama/issues/6286#issuecomment-2307292747) was the script as you supplied, num_ctx=128000. The other iterations were similar in content, some shorter, but looked consistent and complete.

<!-- gh-comment-id:2307963113 --> @rick-github commented on GitHub (Aug 24, 2024): The first run (https://github.com/ollama/ollama/issues/6286#issuecomment-2307292747) was the script as you supplied, `num_ctx=128000`. The other iterations were similar in content, some shorter, but looked consistent and complete.
GiteaMirror commented 2026-04-28 15:55:35 -05:00
Author
Owner
Copy Link

@MeinDeutschkurs commented on GitHub (Aug 24, 2024):

So, this problem seems to be Mac OS specific. As said, with 128000 num_ctx, I get broken (cut off) content. About 1000 tokens, then it stops the output.

I tested it with different 128000 models. it seems to be that this issue is only related to mistral nemo. That's interesting. can Mistral Nemo use more than 128 GB VRAM?

<!-- gh-comment-id:2308313402 --> @MeinDeutschkurs commented on GitHub (Aug 24, 2024): So, this problem seems to be Mac OS specific. As said, with 128000 num_ctx, I get broken (cut off) content. About 1000 tokens, then it stops the output. I tested it with different 128000 models. it seems to be that this issue is only related to mistral nemo. That's interesting. can Mistral Nemo use more than 128 GB VRAM?
GiteaMirror commented 2026-04-28 15:55:38 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 24, 2024):

Have you tried changing num_predict? It's the ollama alias for --predict, set it in the options the same way as num_ctx. I don't see how it would be different for Mac OS but it's worth trying.

<!-- gh-comment-id:2308353630 --> @rick-github commented on GitHub (Aug 24, 2024): Have you tried changing [`num_predict`](https://github.com/ollama/ollama/blob/main/docs/modelfile.md#valid-parameters-and-values:~:text=tfs_z%201-,num_predict,-Maximum%20number%20of)? It's the ollama alias for [--predict](https://github.com/ggerganov/llama.cpp/tree/master/examples/main#number-of-tokens-to-predict), set it in the options the same way as `num_ctx`. I don't see how it would be different for Mac OS but it's worth trying.
GiteaMirror commented 2026-04-28 15:55:39 -05:00
Author
Owner
Copy Link

@MeinDeutschkurs commented on GitHub (Aug 24, 2024):

import argparse
import ollama
from dataclasses import dataclass
from typing import Dict, Callable

@dataclass
class ColorConfig:
    BLUE: str = '\033[94m'
    GREEN: str = '\033[92m'
    YELLOW: str = '\033[93m'
    RED: str = '\033[91m'
    ENDC: str = '\033[0m'

class ColorPrinter:
    def __init__(self, color_config: ColorConfig):
        self.colors = color_config

    def print_colored(self, text: str, color: str):
        color_code = getattr(self.colors, color.upper(), self.colors.ENDC)
        print(f"{color_code}{text}{self.colors.ENDC}")

class TextGenerator:
    def __init__(self, model: str = 'mistral-nemo', options: Dict = None):
        self.model = model
        self.options = options or dict(num_ctx=128000, num_predict=-2)

    def generate(self, prompt: str) -> str:
        response = ollama.generate(model=self.model, prompt=prompt, options=self.options)
        return response['response']

class TextEnhancer:
    def __init__(self, text_generator: TextGenerator, printer: ColorPrinter):
        self.text_generator = text_generator
        self.printer = printer

    def initialize_text(self, initial_prompt: str) -> str:
        current_iteration = self.text_generator.generate(initial_prompt)
        self.printer.print_colored("\n\nInitial text (Iteration 0):", 'BLUE')
        print(f"{current_iteration}\n")
        return current_iteration

    def generate_suggestions(self, initial_prompt: str, current_iteration: str) -> str:
        suggestion_prompt = f"<prompt>{initial_prompt}</prompt> <text>{current_iteration}</text> Suggest how we could make the text better and/or how we can enhance it. Is anything incomplete?"
        return self.text_generator.generate(suggestion_prompt)

    def improve_text(self, initial_prompt: str, current_iteration: str, suggestions: str) -> str:
        improvement_prompt = f"<prompt>{initial_prompt}</prompt><original>{current_iteration}</original> <suggestion>{suggestions}</suggestion> Please use the suggestions to write a better version of original. NO IMAGES/PICTURES/GRAPHICS/LINKS/DIAGRAMS!!"
        return self.text_generator.generate(improvement_prompt)

    def enhance_text(self, initial_prompt: str, loops: int) -> str:
        current_iteration = self.initialize_text(initial_prompt)

        for i in range(loops):
            suggestions = self.generate_suggestions(initial_prompt, current_iteration)
            self.printer.print_colored(f"\n\nSuggestions (Iteration {i+1}):", 'YELLOW')
            print(f"{suggestions}\n")

            current_iteration = self.improve_text(initial_prompt, current_iteration, suggestions)
            self.printer.print_colored(f"\n\nImproved text (Iteration {i+1}):", 'GREEN')
            print(f"{current_iteration}\n")

        return current_iteration

def main():
    parser = argparse.ArgumentParser(description="Text enhancement script")
    parser.add_argument("--prompt", type=str, required=True, help="Initial prompt")
    parser.add_argument("--loops", type=int, default=4, help="Number of enhancement loops")
    parser.add_argument("--model", type=str, default="mistral-nemo", help="Model to use for text generation")
    args = parser.parse_args()

    color_config = ColorConfig()
    printer = ColorPrinter(color_config)
    text_generator = TextGenerator(model=args.model)
    text_enhancer = TextEnhancer(text_generator, printer)

    final_text = text_enhancer.enhance_text(args.prompt, args.loops)
    printer.print_colored("\n\nFinal enhanced text:", 'RED')
    print(final_text)

if __name__ == "__main__":
    main()

I tried num_predict -2 to fill up the entire window.

Observations:

  • It does not use the entire window. (num_predict=-2)
  • If I get English text, everything seems to be fine. The text enhances and expands with each iteration.
  • If I get German text, the text is cut off after about 1000 tokens.

So, num_predict does not solve the issue.

What I can try is to translate the prompt to retrieve the Language from the prompt, translating it to English if necessary and after the loop (if necessary) I have to translate the final output (if necessary).

<!-- gh-comment-id:2308359233 --> @MeinDeutschkurs commented on GitHub (Aug 24, 2024): ``` import argparse import ollama from dataclasses import dataclass from typing import Dict, Callable @dataclass class ColorConfig: BLUE: str = '\033[94m' GREEN: str = '\033[92m' YELLOW: str = '\033[93m' RED: str = '\033[91m' ENDC: str = '\033[0m' class ColorPrinter: def __init__(self, color_config: ColorConfig): self.colors = color_config def print_colored(self, text: str, color: str): color_code = getattr(self.colors, color.upper(), self.colors.ENDC) print(f"{color_code}{text}{self.colors.ENDC}") class TextGenerator: def __init__(self, model: str = 'mistral-nemo', options: Dict = None): self.model = model self.options = options or dict(num_ctx=128000, num_predict=-2) def generate(self, prompt: str) -> str: response = ollama.generate(model=self.model, prompt=prompt, options=self.options) return response['response'] class TextEnhancer: def __init__(self, text_generator: TextGenerator, printer: ColorPrinter): self.text_generator = text_generator self.printer = printer def initialize_text(self, initial_prompt: str) -> str: current_iteration = self.text_generator.generate(initial_prompt) self.printer.print_colored("\n\nInitial text (Iteration 0):", 'BLUE') print(f"{current_iteration}\n") return current_iteration def generate_suggestions(self, initial_prompt: str, current_iteration: str) -> str: suggestion_prompt = f"<prompt>{initial_prompt}</prompt> <text>{current_iteration}</text> Suggest how we could make the text better and/or how we can enhance it. Is anything incomplete?" return self.text_generator.generate(suggestion_prompt) def improve_text(self, initial_prompt: str, current_iteration: str, suggestions: str) -> str: improvement_prompt = f"<prompt>{initial_prompt}</prompt><original>{current_iteration}</original> <suggestion>{suggestions}</suggestion> Please use the suggestions to write a better version of original. NO IMAGES/PICTURES/GRAPHICS/LINKS/DIAGRAMS!!" return self.text_generator.generate(improvement_prompt) def enhance_text(self, initial_prompt: str, loops: int) -> str: current_iteration = self.initialize_text(initial_prompt) for i in range(loops): suggestions = self.generate_suggestions(initial_prompt, current_iteration) self.printer.print_colored(f"\n\nSuggestions (Iteration {i+1}):", 'YELLOW') print(f"{suggestions}\n") current_iteration = self.improve_text(initial_prompt, current_iteration, suggestions) self.printer.print_colored(f"\n\nImproved text (Iteration {i+1}):", 'GREEN') print(f"{current_iteration}\n") return current_iteration def main(): parser = argparse.ArgumentParser(description="Text enhancement script") parser.add_argument("--prompt", type=str, required=True, help="Initial prompt") parser.add_argument("--loops", type=int, default=4, help="Number of enhancement loops") parser.add_argument("--model", type=str, default="mistral-nemo", help="Model to use for text generation") args = parser.parse_args() color_config = ColorConfig() printer = ColorPrinter(color_config) text_generator = TextGenerator(model=args.model) text_enhancer = TextEnhancer(text_generator, printer) final_text = text_enhancer.enhance_text(args.prompt, args.loops) printer.print_colored("\n\nFinal enhanced text:", 'RED') print(final_text) if __name__ == "__main__": main() ``` I tried num_predict -2 to fill up the entire window. Observations: - It does not use the entire window. (num_predict=-2) - If I get English text, everything seems to be fine. The text enhances and expands with each iteration. - If I get German text, the text is cut off after about 1000 tokens. So, num_predict does not solve the issue. What I can try is to translate the prompt to retrieve the Language from the prompt, translating it to English if necessary and after the loop (if necessary) I have to translate the final output (if necessary).
GiteaMirror commented 2026-04-28 15:55:39 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 24, 2024):

So, the problem is only with German text, yet the examples you gave are for English, which seems to be fine?

<!-- gh-comment-id:2308360936 --> @rick-github commented on GitHub (Aug 24, 2024): So, the problem is only with German text, yet the examples you gave are for English, which seems to be fine?
GiteaMirror commented 2026-04-28 15:55:40 -05:00
Author
Owner
Copy Link

@MeinDeutschkurs commented on GitHub (Aug 24, 2024):

Yeah, for any reason, German text is cut off. The English text is as intended. The model claims to handle several languages. I cannot determine why the German generation is cut of after about 1000 tokens.

<!-- gh-comment-id:2308369751 --> @MeinDeutschkurs commented on GitHub (Aug 24, 2024): Yeah, for any reason, German text is cut off. The English text is as intended. The model claims to handle several languages. I cannot determine why the German generation is cut of after about 1000 tokens.
GiteaMirror commented 2026-04-28 15:55:40 -05:00
Author
Owner
Copy Link

@MeinDeutschkurs commented on GitHub (Aug 24, 2024):

Update of the script from above, with automatic language detection:

import argparse
import ollama
from dataclasses import dataclass
from typing import Dict, Tuple

@dataclass
class ColorConfig:
    BLUE: str = '\033[94m'
    GREEN: str = '\033[92m'
    YELLOW: str = '\033[93m'
    RED: str = '\033[91m'
    ENDC: str = '\033[0m'

class ColorPrinter:
    def __init__(self, color_config: ColorConfig):
        self.colors = color_config

    def print_colored(self, text: str, color: str):
        color_code = getattr(self.colors, color.upper(), self.colors.ENDC)
        print(f"{color_code}{text}{self.colors.ENDC}")

class TextGenerator:
    def __init__(self, model: str = 'mistral-nemo', options: Dict = None):
        self.model = model
        self.options = options or dict(num_ctx=128000, num_predict=-2)

    def generate(self, prompt: str) -> str:
        response = ollama.generate(model=self.model, prompt=prompt, options=self.options)
        return response['response']

class LanguageProcessor:
    def __init__(self, text_generator: TextGenerator):
        self.text_generator = text_generator

    def detect_language(self, text: str) -> str:
        prompt = f"Determine the natural language of the following text block and respond with only the language name in English (e.g. English or German or Italian ...): <text>{text}</text>"
        return self.text_generator.generate(prompt).strip()

    def translate_to_english(self, text: str, source_language: str) -> str:
        prompt = f"Translate the following {source_language} text to English: <text>{text}</text>"
        return self.text_generator.generate(prompt)

    def translate_from_english(self, text: str, target_language: str) -> str:
        prompt = f"Translate the following text block to {target_language}. Do not miss anything to translate: <text>{text}</text>"
        return self.text_generator.generate(prompt)

class TextEnhancer:
    def __init__(self, text_generator: TextGenerator, printer: ColorPrinter, language_processor: LanguageProcessor):
        self.text_generator = text_generator
        self.printer = printer
        self.language_processor = language_processor

    def initialize_text(self, initial_prompt: str) -> str:
        current_iteration = self.text_generator.generate(initial_prompt)
        self.printer.print_colored("\n\nInitial text (Iteration 0):", 'BLUE')
        print(f"{current_iteration}\n")
        return current_iteration

    def generate_suggestions(self, initial_prompt: str, current_iteration: str) -> str:
        suggestion_prompt = f"<prompt>{initial_prompt}</prompt> <text>{current_iteration}</text> Suggest how we could make the text better and/or how we can enhance it. Is anything incomplete? DO NOT SUGGEST SOURCES/IMAGES/PICTURES/GRAPHICS/LINKS/DIAGRAMS/HYPERLINKS!!"
        return self.text_generator.generate(suggestion_prompt)

    def improve_text(self, initial_prompt: str, current_iteration: str, suggestions: str) -> str:
        improvement_prompt = f"<prompt>{initial_prompt}</prompt><original>{current_iteration}</original> <suggestion>{suggestions}</suggestion> Please use the suggestions to write a better version of original. NO SOURCES/IMAGES/PICTURES/GRAPHICS/LINKS/DIAGRAMS!!"
        return self.text_generator.generate(improvement_prompt)

    def enhance_text(self, initial_prompt: str, loops: int) -> str:
        current_iteration = self.initialize_text(initial_prompt)
        for i in range(loops):
            suggestions = self.generate_suggestions(initial_prompt, current_iteration)
            self.printer.print_colored(f"\n\nSuggestions (Iteration {i+1}):", 'YELLOW')
            print(f"{suggestions}\n")
            current_iteration = self.improve_text(initial_prompt, current_iteration, suggestions)
            self.printer.print_colored(f"\n\nImproved text (Iteration {i+1}):", 'GREEN')
            print(f"{current_iteration}\n")
        return current_iteration

    def process_and_enhance(self, prompt: str, loops: int) -> Tuple[str, str]:
        original_language = self.language_processor.detect_language(prompt)
        
        if original_language.lower() != 'english':
            self.printer.print_colored(f"\nDetected language: {original_language}. Translating to English...", 'YELLOW')
            english_prompt = self.language_processor.translate_to_english(prompt, original_language)
            self.printer.print_colored(f"\nTranslated prompt: {english_prompt}", 'BLUE')
        else:
            english_prompt = prompt

        enhanced_text = self.enhance_text(english_prompt, loops)

        if original_language.lower() != 'english':
            self.printer.print_colored(f"\nTranslating back to {original_language}...", 'YELLOW')
            final_text = self.language_processor.translate_from_english(enhanced_text, original_language)
        else:
            final_text = enhanced_text

        return final_text, original_language

def main():
    parser = argparse.ArgumentParser(description="Text enhancement script")
    parser.add_argument("--prompt", type=str, required=True, help="Initial prompt")
    parser.add_argument("--loops", type=int, default=4, help="Number of enhancement loops")
    parser.add_argument("--model", type=str, default="mistral-nemo", help="Model to use for text generation")
    args = parser.parse_args()

    color_config = ColorConfig()
    printer = ColorPrinter(color_config)
    text_generator = TextGenerator(model=args.model)
    language_processor = LanguageProcessor(text_generator)
    text_enhancer = TextEnhancer(text_generator, printer, language_processor)

    final_text, original_language = text_enhancer.process_and_enhance(args.prompt, args.loops)

    printer.print_colored(f"\n\nFinal enhanced text ({original_language}):", 'RED')
    print(final_text)

if __name__ == "__main__":
    main()

Tests:

python quicktext.py --prompt "Schreib einen Artikel darüber, wie man ein besserer Opa wird. Benutze bitte Markdown."

python quicktext.py --prompt "Write an article about how to be a better grandpa. Please use Markdown."

<!-- gh-comment-id:2308370167 --> @MeinDeutschkurs commented on GitHub (Aug 24, 2024): Update of the script from above, with automatic language detection: ``` import argparse import ollama from dataclasses import dataclass from typing import Dict, Tuple @dataclass class ColorConfig: BLUE: str = '\033[94m' GREEN: str = '\033[92m' YELLOW: str = '\033[93m' RED: str = '\033[91m' ENDC: str = '\033[0m' class ColorPrinter: def __init__(self, color_config: ColorConfig): self.colors = color_config def print_colored(self, text: str, color: str): color_code = getattr(self.colors, color.upper(), self.colors.ENDC) print(f"{color_code}{text}{self.colors.ENDC}") class TextGenerator: def __init__(self, model: str = 'mistral-nemo', options: Dict = None): self.model = model self.options = options or dict(num_ctx=128000, num_predict=-2) def generate(self, prompt: str) -> str: response = ollama.generate(model=self.model, prompt=prompt, options=self.options) return response['response'] class LanguageProcessor: def __init__(self, text_generator: TextGenerator): self.text_generator = text_generator def detect_language(self, text: str) -> str: prompt = f"Determine the natural language of the following text block and respond with only the language name in English (e.g. English or German or Italian ...): <text>{text}</text>" return self.text_generator.generate(prompt).strip() def translate_to_english(self, text: str, source_language: str) -> str: prompt = f"Translate the following {source_language} text to English: <text>{text}</text>" return self.text_generator.generate(prompt) def translate_from_english(self, text: str, target_language: str) -> str: prompt = f"Translate the following text block to {target_language}. Do not miss anything to translate: <text>{text}</text>" return self.text_generator.generate(prompt) class TextEnhancer: def __init__(self, text_generator: TextGenerator, printer: ColorPrinter, language_processor: LanguageProcessor): self.text_generator = text_generator self.printer = printer self.language_processor = language_processor def initialize_text(self, initial_prompt: str) -> str: current_iteration = self.text_generator.generate(initial_prompt) self.printer.print_colored("\n\nInitial text (Iteration 0):", 'BLUE') print(f"{current_iteration}\n") return current_iteration def generate_suggestions(self, initial_prompt: str, current_iteration: str) -> str: suggestion_prompt = f"<prompt>{initial_prompt}</prompt> <text>{current_iteration}</text> Suggest how we could make the text better and/or how we can enhance it. Is anything incomplete? DO NOT SUGGEST SOURCES/IMAGES/PICTURES/GRAPHICS/LINKS/DIAGRAMS/HYPERLINKS!!" return self.text_generator.generate(suggestion_prompt) def improve_text(self, initial_prompt: str, current_iteration: str, suggestions: str) -> str: improvement_prompt = f"<prompt>{initial_prompt}</prompt><original>{current_iteration}</original> <suggestion>{suggestions}</suggestion> Please use the suggestions to write a better version of original. NO SOURCES/IMAGES/PICTURES/GRAPHICS/LINKS/DIAGRAMS!!" return self.text_generator.generate(improvement_prompt) def enhance_text(self, initial_prompt: str, loops: int) -> str: current_iteration = self.initialize_text(initial_prompt) for i in range(loops): suggestions = self.generate_suggestions(initial_prompt, current_iteration) self.printer.print_colored(f"\n\nSuggestions (Iteration {i+1}):", 'YELLOW') print(f"{suggestions}\n") current_iteration = self.improve_text(initial_prompt, current_iteration, suggestions) self.printer.print_colored(f"\n\nImproved text (Iteration {i+1}):", 'GREEN') print(f"{current_iteration}\n") return current_iteration def process_and_enhance(self, prompt: str, loops: int) -> Tuple[str, str]: original_language = self.language_processor.detect_language(prompt) if original_language.lower() != 'english': self.printer.print_colored(f"\nDetected language: {original_language}. Translating to English...", 'YELLOW') english_prompt = self.language_processor.translate_to_english(prompt, original_language) self.printer.print_colored(f"\nTranslated prompt: {english_prompt}", 'BLUE') else: english_prompt = prompt enhanced_text = self.enhance_text(english_prompt, loops) if original_language.lower() != 'english': self.printer.print_colored(f"\nTranslating back to {original_language}...", 'YELLOW') final_text = self.language_processor.translate_from_english(enhanced_text, original_language) else: final_text = enhanced_text return final_text, original_language def main(): parser = argparse.ArgumentParser(description="Text enhancement script") parser.add_argument("--prompt", type=str, required=True, help="Initial prompt") parser.add_argument("--loops", type=int, default=4, help="Number of enhancement loops") parser.add_argument("--model", type=str, default="mistral-nemo", help="Model to use for text generation") args = parser.parse_args() color_config = ColorConfig() printer = ColorPrinter(color_config) text_generator = TextGenerator(model=args.model) language_processor = LanguageProcessor(text_generator) text_enhancer = TextEnhancer(text_generator, printer, language_processor) final_text, original_language = text_enhancer.process_and_enhance(args.prompt, args.loops) printer.print_colored(f"\n\nFinal enhanced text ({original_language}):", 'RED') print(final_text) if __name__ == "__main__": main() ``` Tests: `python quicktext.py --prompt "Schreib einen Artikel darüber, wie man ein besserer Opa wird. Benutze bitte Markdown."` `python quicktext.py --prompt "Write an article about how to be a better grandpa. Please use Markdown."`
GiteaMirror commented 2026-04-28 15:55:41 -05:00
Author
Owner
Copy Link

@MeinDeutschkurs commented on GitHub (Aug 24, 2024):

Even here, the final (German) text is cut off.

<!-- gh-comment-id:2308403628 --> @MeinDeutschkurs commented on GitHub (Aug 24, 2024): Even here, the final (German) text is cut off.
GiteaMirror commented 2026-04-28 15:55:41 -05:00
Author
Owner
Copy Link

@MeinDeutschkurs commented on GitHub (Aug 26, 2024):

So, the problem is only with German text, yet the examples you gave are for English, which seems to be fine?

There should be no problem, independent from the language... is there a problem with the tokenizer?

<!-- gh-comment-id:2309216025 --> @MeinDeutschkurs commented on GitHub (Aug 26, 2024): > So, the problem is only with German text, yet the examples you gave are for English, which seems to be fine? There should be no problem, independent from the language... is there a problem with the tokenizer?
GiteaMirror commented 2026-04-28 15:55:42 -05:00
Author
Owner
Copy Link

@neorock07 commented on GitHub (Oct 17, 2024):

This approach could fix my issue, the default of n_ctx in Ollama is 2048 token either I use small model (2b) or medium model (9b).
in my case I tried to use Llama 3.1 and Gemma2, this is how to enlarge token via modelfile file.

  1. open terminal
  2. pull your prefered model
  3. then, type in your terminal this (example gemma2, change with your model):
    ollama show gemma2:9b --modelfile > gemma.modelfile
  4. open that file by using nano/pico : nano gemma.modelfile
  5. scroll untill you find :

.{{ .Response }}<end_of_turn>

PARAMETER stop <start_of_turn>
PARAMETER stop <end_of_turn>
LICENSE Gemma Terms of Use

Last modified: February 21, 2024...

,
Or in llama just scroll untill you find key word "Parameter" then add this code :
PARAMETER num_ctx 8192 below "PARAMETER stop <end_of_turn>" .

  1. finally we create new model file base on our custom modelfile:
    ollama create gemma_8192 --file gemma.modelfile

  2. next just check out our new model with :
    ollama list

NAME ID SIZE MODIFIED
gemma_8192:latest 30ed6114610b 5.4 GB 2 minutes ago
gemma2:9b ff02c3702f32 5.4 GB 4 minutes ago

that's it just go and use that model, I actually could change n_ctx to 8192 length.

<!-- gh-comment-id:2418807319 --> @neorock07 commented on GitHub (Oct 17, 2024): This approach could fix my issue, the default of n_ctx in Ollama is 2048 token either I use small model (2b) or medium model (9b). in my case I tried to use Llama 3.1 and Gemma2, this is how to enlarge token via modelfile file. 1. open terminal 2. pull your prefered model 3. then, type in your terminal this (example gemma2, change with your model): `ollama show gemma2:9b --modelfile > gemma.modelfile` 4. open that file by using nano/pico : `nano gemma.modelfile` 5. scroll untill you find : > .{{ .Response }}<end_of_turn> > > PARAMETER stop <start_of_turn> > PARAMETER stop <end_of_turn> > LICENSE Gemma Terms of Use > > Last modified: February 21, 2024... , Or in llama just scroll untill you find key word "Parameter" then add this code : `PARAMETER num_ctx 8192` below "PARAMETER stop <end_of_turn>" . 6. finally we create new model file base on our custom modelfile: `ollama create gemma_8192 --file gemma.modelfile` 7. next just check out our new model with : `ollama list` > NAME ID SIZE MODIFIED > gemma_8192:latest 30ed6114610b 5.4 GB 2 minutes ago > gemma2:9b ff02c3702f32 5.4 GB 4 minutes ago that's it just go and use that model, I actually could change n_ctx to 8192 length.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#50449
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?