Chiedo se qualcuno è riuscito a farlo funzionare con Ollama in locale?
Sto anche cercando di far funzionare questo con ollama, sembra che l’API utilizzata dal plugin Discourse AI non sia compatibile con Ollama e qualsiasi impostazione io modifichi non cambia la richiesta API.
Questo è ciò che funziona con ollama:
curl http://192.168.1.2:11434/api/generate -d '{"model": "llama3.2", "prompt": "Perché il cielo è blu?"}'
E sopra sembra essere l’unico tipo di richiesta che il plugin tenta. Ad esempio:
curl http://192.168.1.2:11434/ \
-X POST \
-H 'Content-Type: application/json' \
-d '{"inputs":"<s>[INST] Qual è il tuo condimento preferito? [/INST] Beh, sono piuttosto affezionato a una buona spruzzata di succo di limone fresco. Aggiunge la giusta quantità di sapore acidulo a tutto ciò che cucino in cucina!</s> [INST] Hai ricette di maionese? [/INST]","parameters":{"max_new_tokens":500, "temperature":0.5,"top_p": 0.9}}'
Il chatbot AI è presuntamente compatibile con ollama ma non riesco a ottenere una risposta nemmeno da quello.
Quindi, chiunque abbia ottenuto questo risultato con ollama, per favore posti le impostazioni che ha utilizzato!
2 Mi Piace
Apparentemente ho parlato troppo presto, se non ho alcuna impostazione nel plugin Discourse AI impostata e seguo le istruzioni RAG in questo post ottengo risposte nella chat bot AI da Ollama!
1 Mi Piace
Funziona abbastanza bene con Ollama, abbiamo più persone nel team che lo utilizzano.
1 Mi Piace
Fantastico. Esiste una guida che possiamo seguire su come implementarlo?
Sono più interessato alla sintesi e ad altre funzioni che al chatbot.
ETA: Questa configurazione non funziona:
(Ho oscurato l’indirizzo IP del mio host qui).
Quello che succede è che ricevo un Internal Server Error quando clicco su “Run Test”.
Su /logs, vedo:
NameError (variabile locale o metodo `tokenizer' non definito per un'istanza di DiscourseAi::Completions::Dialects::ChatGpt)
app/controllers/application_controller.rb:427:in `block in with_resolved_locale'
app/controllers/application_controller.rb:427:in `with_resolved_locale'
lib/middleware/omniauth_bypass_middleware.rb:35:in `call'
lib/middleware/csp_script_nonce_injector.rb:12:in `call'
lib/middleware/anonymous_cache.rb:409:in `call'
lib/middleware/csp_script_nonce_injector.rb:12:in `call'
config/initializers/008-rack-cors.rb:14:in `call'
config/initializers/100-quiet_logger.rb:20:in `call'
config/initializers/100-silence_logger.rb:29:in `call'
lib/middleware/enforce_hostname.rb:24:in `call'
lib/middleware/processing_request.rb:12:in `call'
lib/middleware/request_tracker.rb:385:in `call'
Questo accade indipendentemente dal tokenizer che seleziono. Sto testando questo su 3.5.0beta1-dev (c1ee4e120e).
1 Mi Piace
Imposta l’URL su http://localhost:11434/v1/chat/completions
1 Mi Piace
Il test continua a fallire con un errore del server interno, stesso messaggio di log: un NameError che indica che ‘tokenizer’ è una variabile locale non definita.
1 Mi Piace
Prova a salvare il modello così com’è, aggiornare la pagina e poi tornare tramite il pulsante di modifica LLM ed eseguire un nuovo test.
Stesso risultato: l’ho ripetuto alcune volte, ma ci ho riprovato adesso.
Ho eseguito l’installazione nel modo consueto: l’ho aggiunto al mio file app.yml e ho eseguito ./launcher rebuild app
La riga che ho aggiunto è stata:
hooks:
after_code:
- exec:
cd: $home/plugins
cmd:
- sudo -E -u discourse git clone https://github.com/discourse/discourse-ai.git
(Ho rimosso gli altri plugin dall’elenco, ma ho lasciato i componenti yml in modo che tu possa vedere dove si trova questo nel file yml.)
1 Mi Piace
Domani ripristinerò la mia configurazione locale di ollama e segnalerò qui.
1 Mi Piace
Ho fatto:
ollama serve
ollama run phi4
# test
curl http://localhost:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "phi4",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "How much is 1+1?"
}
]
}' -s | jq .choices[].message.content
"La somma di 1 + 1 è 2.
Nell'aritmetica di base, quando si aggiunge un'unità a un'altra singola unità, il numero totale diventa due unità. Questa operazione fa parte dei principi fondamentali di addizione utilizzati in matematica. C'è qualcos'altro in cui posso aiutarti riguardo alla matematica o a qualsiasi altro argomento?"
Testato su Helper
2 Mi Piace
Thanks for testing this.
I notice that I’ve got an extra option (Disable streaming completions) compared to your config. Is it possible it’s the model selection that’s causing an issue for me? (I’ve got deepseek-r1:32b installed).
curl http://localhost:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "deepseek-r1:32b",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "How much is 1+1?"
}
]
}' -s | jq .choices[].message.content
"<think>\nAlright, I'm trying to figure out how much 1 plus 1 equals. Okay, so let's break this down step by step even though it seems simple.\n\nFirst, I know that addition is combining two numbers to get a total sum. So, when we say 1 + 1, we're essentially putting together one object and another object to see how many we have in total.\n\nLet me visualize it: imagine I have one apple in my left hand and another apple in my right hand. If I put them together, how many apples do I have? It should be two apples, right?\n\nAnother way to think about it is using a number line. Starting at the number 1, if I move forward by one more step, where do I land? That's right, on the number 2.\n\nI can also use my fingers to count. Hold up one finger on my left hand and another on my right. Now, count them all: one, two. So that gives me a total of two fingers, which means 1 + 1 = 2.\n\nMaybe I should check if there's another perspective or method. For instance, in binary, the number system used by computers, 1 + 1 equals 10 because it carries over after reaching 2 (which is represented as 10). But that's a different context. In standard base-10 arithmetic, which we use daily, 1 + 1 remains 2.\n\nLet me also consider the concept of sets. If I have a set containing one element and another set with one element, combining them would result in a set with two elements. That reinforces that 1 + 1 equals 2.\n\nI might wonder if there's any scenario where 1 + 1 doesn't equal 2. But within the realm of basic arithmetic and mathematics as we learn it, the answer is consistently two.\n\nTo make sure I'm not missing anything, maybe I should look into some mathematical proofs or principles that confirm this. For example, in Peano axioms, which form the foundation of arithmetic, the idea of adding 1 to 1 would follow the axiom that every natural number has a successor, and 2 is defined as the successor of 1.\n\nTherefore, putting all these thoughts together—visualizing objects, using a number line, counting fingers, considering sets, and even touching on mathematical axioms—it all points towards 1 + 1 equaling 2.\n</think>\n\nThe result of adding 1 and 1 together is 2. This can be seen in simple arithmetic and various ways of visualizing the problem.\n\n**Answer:** \n$1 + 1 = \\boxed{2}$"
Similar result, though I get the <think> tags with this particular model - so maybe the model won’t be suitable for this use.
I’ve just rebuilt (ie, ./launcher rebuild app) my installation and made sure everything is current - I show “all up-to-date” in my Update page.
Selecting either deepseek-r1:32b or phi4 (which I installed and tested as well), I still am getting the internal server error response in the browser.
But in my logs, I’m seeing something new.
FinalDestination::SSRFDetector::DisallowedIpError (FinalDestination: all resolved IPs were disallowed) lib/final_destination/ssrf_detector.rb:105:in 'lookup_and_filter_ips' lib/final_destination/http'
lib/final_destination/ssrf_detector.rb:105:in `lookup_and_filter_ips'
lib/final_destination/http.rb:15:in `connect'
net-http (0.6.0) lib/net/http.rb:1642:in `do_start'
net-http (0.6.0) lib/net/http.rb:1631:in `start'
net-http (0.6.0) lib/net/http.rb:1070:in `start'
plugins/discourse-ai/lib/completions/endpoints/base.rb:105:in `perform_completion!'
plugins/discourse-ai/lib/completions/endpoints/open_ai.rb:44:in `perform_completion!'
plugins/discourse-ai/lib/completions/llm.rb:281:in `generate'
plugins/discourse-ai/lib/configuration/llm_validator.rb:36:in `run_test'
plugins/discourse-ai/app/controllers/discourse_ai/admin/ai_llms_controller.rb:128:in `test'
actionpack (7.2.2.1) lib/action_controller/metal/basic_implicit_render.rb:8:in `send_action'
actionpack (7.2.2.1) lib/abstract_controller/base.rb:226:in `process_action'
actionpack (7.2.2.1) lib/action_controller/metal/rendering.rb:193:in `process_action'
actionpack (7.2.2.1) lib/abstract_controller/callbacks.rb:261:in `block in process_action'
activesupport (7.2.2.1) lib/active_support/callbacks.rb:121:in `block in run_callbacks'
app/controllers/application_controller.rb:427:in `block in with_resolved_locale'
i18n (1.14.7) lib/i18n.rb:353:in `with_locale'
app/controllers/application_controller.rb:427:in `with_resolved_locale'
activesupport (7.2.2.1) lib/active_support/callbacks.rb:130:in `block in run_callbacks'
activesupport (7.2.2.1) lib/active_support/callbacks.rb:141:in `run_callbacks'
actionpack (7.2.2.1) lib/abstract_controller/callbacks.rb:260:in `process_action'
actionpack (7.2.2.1) lib/action_controller/metal/rescue.rb:27:in `process_action'
actionpack (7.2.2.1) lib/action_controller/metal/instrumentation.rb:77:in `block in process_action'
activesupport (7.2.2.1) lib/active_support/notifications.rb:210:in `block in instrument'
activesupport (7.2.2.1) lib/active_support/notifications/instrumenter.rb:58:in `instrument'
activesupport (7.2.2.1) lib/active_support/notifications.rb:210:in `instrument'
actionpack (7.2.2.1) lib/action_controller/metal/instrumentation.rb:76:in `process_action'
actionpack (7.2.2.1) lib/action_controller/metal/params_wrapper.rb:259:in `process_action'
activerecord (7.2.2.1) lib/active_record/railties/controller_runtime.rb:39:in `process_action'
actionpack (7.2.2.1) lib/abstract_controller/base.rb:163:in `process'
actionview (7.2.2.1) lib/action_view/rendering.rb:40:in `process'
rack-mini-profiler (3.3.1) lib/mini_profiler/profiling_methods.rb:115:in `block in profile_method'
actionpack (7.2.2.1) lib/action_controller/metal.rb:252:in `dispatch'
actionpack (7.2.2.1) lib/action_controller/metal.rb:335:in `dispatch'
actionpack (7.2.2.1) lib/action_dispatch/routing/route_set.rb:67:in `dispatch'
actionpack (7.2.2.1) lib/action_dispatch/routing/route_set.rb:50:in `serve'
actionpack (7.2.2.1) lib/action_dispatch/routing/mapper.rb:32:in `block in <class:Constraints>'
actionpack (7.2.2.1) lib/action_dispatch/routing/mapper.rb:62:in `serve'
actionpack (7.2.2.1) lib/action_dispatch/journey/router.rb:53:in `block in serve'
actionpack (7.2.2.1) lib/action_dispatch/journey/router.rb:133:in `block in find_routes'
actionpack (7.2.2.1) lib/action_dispatch/journey/router.rb:126:in `each'
actionpack (7.2.2.1) lib/action_dispatch/journey/router.rb:126:in `find_routes'
actionpack (7.2.2.1) lib/action_dispatch/journey/router.rb:34:in `serve'
actionpack (7.2.2.1) lib/action_dispatch/routing/route_set.rb:896:in `call'
lib/middleware/omniauth_bypass_middleware.rb:35:in `call'
rack (2.2.11) lib/rack/tempfile_reaper.rb:15:in `call'
rack (2.2.11) lib/rack/conditional_get.rb:27:in `call'
rack (2.2.11) lib/rack/head.rb:12:in `call'
actionpack (7.2.2.1) lib/action_dispatch/http/permissions_policy.rb:38:in `call'
lib/content_security_policy/middleware.rb:12:in `call'
lib/middleware/anonymous_cache.rb:409:in `call'
lib/middleware/csp_script_nonce_injector.rb:12:in `call'
config/initializers/008-rack-cors.rb:14:in `call'
rack (2.2.11) lib/rack/session/abstract/id.rb:266:in `context'
rack (2.2.11) lib/rack/session/abstract/id.rb:260:in `call'
actionpack (7.2.2.1) lib/action_dispatch/middleware/cookies.rb:704:in `call'
actionpack (7.2.2.1) lib/action_dispatch/middleware/callbacks.rb:31:in `block in call'
activesupport (7.2.2.1) lib/active_support/callbacks.rb:101:in `run_callbacks'
actionpack (7.2.2.1) lib/action_dispatch/middleware/callbacks.rb:30:in `call'
actionpack (7.2.2.1) lib/action_dispatch/middleware/debug_exceptions.rb:31:in `call'
actionpack (7.2.2.1) lib/action_dispatch/middleware/show_exceptions.rb:32:in `call'
logster (2.20.1) lib/logster/middleware/reporter.rb:40:in `call'
railties (7.2.2.1) lib/rails/rack/logger.rb:41:in `call_app'
railties (7.2.2.1) lib/rails/rack/logger.rb:29:in `call'
config/initializers/100-quiet_logger.rb:20:in `call'
config/initializers/100-silence_logger.rb:29:in `call'
actionpack (7.2.2.1) lib/action_dispatch/middleware/request_id.rb:33:in `call'
lib/middleware/enforce_hostname.rb:24:in `call'
rack (2.2.11) lib/rack/method_override.rb:24:in `call'
actionpack (7.2.2.1) lib/action_dispatch/middleware/executor.rb:16:in `call'
rack (2.2.11) lib/rack/sendfile.rb:110:in `call'
rack-mini-profiler (3.3.1) lib/mini_profiler.rb:334:in `call'
lib/middleware/processing_request.rb:12:in `call'
message_bus (4.3.9) lib/message_bus/rack/middleware.rb:60:in `call'
lib/middleware/request_tracker.rb:385:in `call'
actionpack (7.2.2.1) lib/action_dispatch/middleware/remote_ip.rb:96:in `call'
railties (7.2.2.1) lib/rails/engine.rb:535:in `call'
railties (7.2.2.1) lib/rails/railtie.rb:226:in `public_send'
railties (7.2.2.1) lib/rails/railtie.rb:226:in `method_missing'
rack (2.2.11) lib/rack/urlmap.rb:74:in `block in call'
rack (2.2.11) lib/rack/urlmap.rb:58:in `each'
rack (2.2.11) lib/rack/urlmap.rb:58:in `call'
unicorn (6.1.0) lib/unicorn/http_server.rb:634:in `process_client'
unicorn (6.1.0) lib/unicorn/http_server.rb:739:in `worker_loop'
unicorn (6.1.0) lib/unicorn/http_server.rb:547:in `spawn_missing_workers'
unicorn (6.1.0) lib/unicorn/http_server.rb:143:in `start'
unicorn (6.1.0) bin/unicorn:128:in `<top (required)>'
vendor/bundle/ruby/3.3.0/bin/unicorn:25:in `load'
vendor/bundle/ruby/3.3.0/bin/unicorn:25:in `<main>'
This Discourse install is realtively fresh, and I’ve not configured any IP address blocking. I’m also the only user on it. I am running ollama in a Docker container, but that shouldn’t be an issue; I use open-webui (also from another docker container) with the instance and that works fine. I am using the local IP address rather than “localhost”, but “localhost” doesn’t behave any differently.
Appreciate the time you’re putting into help figure out what’s going on here.
ETA: I installed jq in the Discourse docker container, and ran the curl command (substituting my local IP address for localhost), and it did run from there - so there’s connectivity between the containers, and the URL is correct).
Questa è la nostra protezione SSRF predefinita che entra in funzione. Puoi aggiungere eccezioni tramite la variabile d’ambiente DISCOURSE_ALLOWED_INTERNAL_HOSTS.
Si attiva quando Discourse tenta di raggiungere un IP considerato interno.
4 Mi Piace
Fantastico, ha funzionato: l’ho fatto tramite le impostazioni invece che tramite la variabile ENV, ma il test ora è passato. Grazie ancora per tutto l’aiuto!
5 Mi Piace
3 post sono stati divisi in un nuovo argomento: Come usare Ollama con gli embedding
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.