"Net::HTTPBadResponse" errors on Gemini Embeddings

Bug
,
1

I’m using Gemini for the LLM, I’m noticing a lot of errors in the logs and it appears to be coming from a embeddings backfill task.

Job exception: Net::HTTPBadResponse

/var/www/discourse/plugins/discourse-ai/app/jobs/scheduled/embeddings_backfill.rb:122:in `populate_topic_embeddings’

Based on the architecture diagram above, it should only run on a new topic but it appears that there’s a backfill task which runs on existing topics as well. That in itself it actually a good thing since it helps existing topics as well.

My clarification is related to limits. If Gemini is limited to 60 queries per minute, I suspect that the threshold may be causing the errors (is there a way to confirm?).
When a threshold is hit, does discourse retry to embeddings for the failed topic after a while? If so, how often and for how long? If not, is there a way to manually trigger the backfill task for related topics to go back and handle the unprocessed topics?

1 Like
2

You can adjust your backfill rate using the hidden site setting ai_embeddings_backfill_batch_size. You can set it to 50 to comply with free Gemini limits.

1 Like
3

Thanks. I just did it, I’m assuming the app doesn’t need to be restarted. Perhaps it may be a good idea to expose this settings in the admin panel under the AI settings page as I suspect different models may provide tiered limits.

Can I assume that the AI module keeps trying to backfill unprocessed topics indefinitely without any intervention if it hits a limit or fails for whatever reason (e.g. connectivity or loss of authentication)?

1 Like
4

Unfortunately that doesn’t seem to be working. I’m still seeing the number of errors Job exception: Net::HTTPBadResponse going on increasing. I’ve tried to restart the system, verified in rails the number backfill batch is set to 50. What else can I do/check here?

5

Yes

What’s the output of running the command below with your API key?

curl https://generativelanguage.googleapis.com/v1beta/models/embedding-001:embedContent?key=API_KEY_GOES_HERE -H 'Content-Type: application/json'     -X POST     -d '{ "content": { "parts":[{ "text": "Why use Discourse for my online community?"}]}}'
1 Like
6 
{
  "embedding": {
    "values": [
      0.049835667,
      -0.053087797,
      -0.0026651944,
      -0.032271475,
      0.0008565384,
      0.012963673,
      0.04925958,
      0.02114348,
      0.017131783,
      0.07987728,
      0.00036036532,
      0.014957308,
      0.004499466,
      -0.031672746,
      0.045119844,
      -0.043746613,
      0.013044078,
      -0.015635632,
      -0.0034702981,
      0.00055249245,
      -0.043857947,
      -0.02738547,
      -0.0023206542,
      -0.043359935,
      0.018656434,
      -0.016405085,
      -0.025446353,
      -0.09162045,
      -0.009193362,
      0.030849759,
      -0.07839411,
      0.073293,
      -0.0053825686,
      -0.0059678303,
      -0.0006844548,
      -0.006315334,
      0.032252856,
      0.006667941,
      -0.027604295,
      0.05220841,
      -0.048274767,
      -0.029392688,
      -0.029355835,
      0.04653804,
      0.003945584,
      -0.043296136,
      -0.015069529,
      -0.023561781,
      0.029110815,
      -0.032166,
      0.04572339,
      0.029018285,
      0.048896253,
      -0.021334046,
      0.03946595,
      -0.0169248,
      0.016381577,
      -0.012894736,
      -0.05581367,
      -0.014831158,
      0.030060403,
      -0.008313037,
      -0.09008968,
      0.04903514,
      0.041317243,
      0.0027498198,
      -0.031350896,
      -0.0081006335,
      0.0390009,
      0.01456084,
      0.0315405,
      0.030226065,
      0.07349099,
      -0.052606054,
      -0.023721961,
      -0.053736396,
      -0.0710476,
      0.07822372,
      -0.0048553,
      -0.004152618,
      -0.0069499593,
      -0.03163849,
      -0.038669657,
      -0.03751683,
      -0.056960687,
      0.06533119,
      -0.057827037,
      0.01080756,
      0.017270816,
      0.034793865,
      -0.021642657,
      -0.01865893,
      -0.0056911353,
      -0.045503955,
      0.039082598,
      0.08366713,
      -0.007643605,
      -0.03659021,
      0.015616355,
      -0.003906713,
      -0.0076599843,
      -0.036598705,
      -0.01108682,
      -0.0069597336,
      0.05237777,
      0.06565515,
      0.027545689,
      0.06075108,
      -0.048622813,
      0.024223646,
      -0.071117215,
      0.034065973,
      -0.03702431,
      -0.017262084,
      0.010666043,
      -0.018599784,
      0.011046217,
      0.09575254,
      -0.010923603,
      -0.005287933,
      0.07443102,
      0.015511241,
      0.02358482,
      0.0038122602,
      0.0110562965,
      0.06479849,
      0.006115338,
      0.02237977,
      0.055337675,
      0.056075558,
      0.0025006428,
      -0.015629748,
      0.0009295725,
      0.053074196,
      0.08249292,
      0.06543386,
      0.019779298,
      0.0106866425,
      0.005572269,
      -0.0023958154,
      -0.0037621462,
      0.020007739,
      0.0139754675,
      0.00069694134,
      -0.06502769,
      0.022389991,
      -0.05675248,
      -0.04237161,
      0.0069264187,
      -0.043192238,
      -0.037506286,
      0.036144827,
      -0.04514279,
      0.00876093,
      0.041715626,
      -0.010649575,
      0.011852339,
      0.05341852,
      -0.012773308,
      -0.038706943,
      0.03152377,
      0.05521588,
      -0.00361507,
      -0.018567959,
      -0.012679102,
      -0.029120263,
      -0.026539568,
      0.00030241,
      -0.008183379,
      -0.03906954,
      -0.020941496,
      -0.0032074258,
      -0.026327813,
      -0.03912452,
      -0.027052578,
      -0.040369555,
      0.008954807,
      0.010244666,
      -0.0025794536,
      0.025502704,
      -0.07240107,
      -0.036659658,
      -0.010231007,
      0.016302444,
      0.042812273,
      -0.002724102,
      0.07421903,
      -0.10142466,
      -0.029415531,
      -0.043778807,
      0.016563682,
      -0.0270507,
      -0.000113599424,
      0.0033279394,
      -0.016658602,
      -0.021675194,
      0.05576477,
      0.06862853,
      -0.00652585,
      -0.01159096,
      0.012575702,
      0.10570678,
      0.054589298,
      -0.009304841,
      0.08004885,
      0.007919161,
      0.07728513,
      -0.041437797,
      -0.01067075,
      0.034822658,
      0.0018481511,
      -0.0014734427,
      0.0048331176,
      0.022597842,
      0.048678137,
      0.091887526,
      -0.020750577,
      0.019252019,
      0.030849814,
      -0.041854903,
      -0.014453777,
      -0.021936258,
      -0.014346192,
      0.0115072755,
      -0.052798618,
      0.04037987,
      0.0006653009,
      0.056538783,
      -0.007427154,
      -0.06047292,
      0.011042166,
      0.060032465,
      0.00742375,
      -0.034923185,
      0.040616427,
      0.010072564,
      -0.017704858,
      -0.039070953,
      -0.011488165,
      0.006652084,
      -0.041514922,
      0.03861204,
      0.013663087,
      0.08342337,
      -0.073423214,
      -0.0054284534,
      -0.0058582253,
      0.014006357,
      -0.0038960788,
      0.07571122,
      -0.0048305513,
      -0.031443343,
      0.052626867,
      0.0022661658,
      -0.08984796,
      0.0011647721,
      -0.035565842,
      -0.0049919626,
      -0.03326319,
      0.039390538,
      0.018988734,
      -0.041907478,
      0.01740078,
      -0.03847038,
      -0.018651431,
      -0.034888804,
      -0.02135333,
      -0.02415797,
      -0.005345686,
      0.027105553,
      -0.014054851,
      -0.047500543,
      0.09427313,
      -0.013695819,
      -0.01498477,
      0.053596623,
      0.012254832,
      0.04908039,
      -0.029091777,
      -0.07372432,
      0.0343596,
      0.02258552,
      -0.0071253153,
      -0.02386305,
      -0.0057186238,
      -0.03323554,
      -0.029094461,
      -0.032949373,
      0.059173636,
      -0.046209954,
      -0.015498728,
      -0.024551516,
      0.011513828,
      -0.061083652,
      -0.02237468,
      -0.0049111806,
      -0.012389059,
      0.028721081,
      0.04778319,
      0.0073474767,
      -0.0065232418,
      -0.014830325,
      -0.028026134,
      -0.0653796,
      0.019258047,
      0.024961341,
      0.03533581,
      -0.030846452,
      0.06656185,
      -0.015825864,
      0.027694814,
      -0.017256135,
      0.010215107,
      0.0050955433,
      0.01311972,
      0.037823886,
      -0.026694939,
      0.050253987,
      -0.002114194,
      0.06720399,
      0.019737212,
      0.030683005,
      -0.016445823,
      0.023714663,
      0.025072183,
      0.01696251,
      -0.029809875,
      0.012283688,
      0.0008943514,
      -0.00018241217,
      -0.031002654,
      0.030267857,
      0.01593443,
      0.02385287,
      -0.023456587,
      -0.0017674654,
      -0.069804,
      0.042437166,
      -0.0074308217,
      0.071244255,
      0.014429643,
      0.006895311,
      0.01402888,
      -0.065092355,
      -0.0110172685,
      0.013090774,
      -0.008232141,
      0.0068813222,
      0.059676517,
      0.050064728,
      0.016562462,
      0.06436298,
      -0.045705132,
      0.009147997,
      0.0068886406,
      -0.028144563,
      0.040914834,
      -0.030774482,
      0.032046646,
      -0.06756744,
      -0.087018155,
      0.032553025,
      -0.013287733,
      0.006165214,
      0.027844446,
      -0.0454315,
      0.0007838677,
      -0.02824797,
      -0.012193629,
      0.0052360683,
      0.026328225,
      -0.031589665,
      0.031567544,
      -5.823923e-05,
      -0.025981033,
      -0.057981066,
      -0.023402635,
      -0.016856203,
      0.016395364,
      -0.007495069,
      -0.04980727,
      -0.013136656,
      0.011539791,
      0.039624084,
      0.029931411,
      0.008295852,
      -0.0052006287,
      0.027126353,
      0.041326083,
      -0.030452484,
      0.023171015,
      0.027159497,
      0.061324485,
      -0.013437325,
      0.026905138,
      -0.015175969,
      0.001190477,
      -0.03296332,
      0.03607256,
      -0.021430602,
      -0.00838613,
      -0.03176932,
      -0.03841174,
      -0.005889474,
      -0.031237358,
      -0.029592663,
      -0.009253579,
      -0.027543431,
      -0.013652089,
      -0.0122762155,
      -0.0051506646,
      0.030370936,
      0.034951918,
      -0.025405152,
      0.00054425595,
      -0.009923217,
      0.025618095,
      -0.025864856,
      0.048924446,
      0.032274988,
      0.02552666,
      -0.06287375,
      0.03265969,
      0.028939355,
      0.012768615,
      -0.060623392,
      -0.06054021,
      -0.018593578,
      0.01658346,
      0.009148355,
      0.032717086,
      0.025887506,
      0.005102726,
      -0.024044968,
      0.015867848,
      -0.06985226,
      0.022473859,
      0.029198112,
      -0.022219934,
      -0.015967099,
      0.046363156,
      -0.033796955,
      0.019713182,
      -0.03798754,
      -0.047272854,
      -0.0033509203,
      0.0046040253,
      -0.018489538,
      0.039789207,
      -0.040447205,
      0.042606983,
      -0.07086798,
      -0.029225757,
      -0.074334815,
      -0.011229469,
      -0.07036015,
      0.028998882,
      0.031645387,
      -0.054167435,
      -0.011917344,
      -0.054065116,
      -0.039588124,
      -0.036324322,
      -0.082037084,
      -0.01200532,
      0.019225745,
      0.024985975,
      -0.0042101457,
      0.0127587365,
      0.04495904,
      -0.017835576,
      -0.0057440917,
      -0.002545832,
      -0.0440076,
      0.017551765,
      -0.0010223053,
      -0.084200606,
      0.026969673,
      -0.023898808,
      -0.010268309,
      0.039836444,
      -0.016278664,
      0.0067669223,
      0.00091878435,
      0.01744076,
      -0.036685348,
      -0.028350586,
      -0.0075139287,
      -0.04298965,
      0.03240625,
      0.013864248,
      -0.020742482,
      -0.02164198,
      -0.0614275,
      -0.021606905,
      0.07318286,
      -0.0147202425,
      0.009992278,
      0.018593978,
      0.029263247,
      -0.013064598,
      -0.029978039,
      0.003842346,
      -0.059589136,
      0.04579801,
      -0.005114194,
      0.035829544,
      0.0035099334,
      0.00045971398,
      -0.01986417,
      -0.05324282,
      0.018024689,
      0.020279268,
      -0.015164831,
      0.038371563,
      -0.025026957,
      -0.0076083476,
      0.008192915,
      0.011792553,
      0.014137991,
      0.0051719313,
      0.01869812,
      -0.09118679,
      0.011270221,
      0.008047813,
      -0.07419907,
      0.06346611,
      0.049479857,
      -0.06060518,
      0.03438583,
      0.0153429685,
      0.07922743,
      -0.03877446,
      0.006707378,
      -0.008108458,
      -0.046597168,
      0.0054507926,
      0.021386953,
      -0.009807047,
      0.020110589,
      0.012233969,
      0.015837837,
      0.0037049693,
      0.039532676,
      0.017257283,
      0.00886057,
      0.034195136,
      -0.09243383,
      0.053717166,
      -0.0021029164,
      -0.06875445,
      -0.0071924697,
      0.03108134,
      -0.04563619,
      -0.00083774194,
      0.014210709,
      -0.060640357,
      0.04132176,
      0.009436319,
      0.026991349,
      -0.019189715,
      -0.014624527,
      -0.018609596,
      -0.003482946,
      0.027900012,
      0.06637678,
      -0.0012586409,
      -0.038126424,
      0.02849398,
      -0.018801333,
      -0.0042140875,
      -0.00043351148,
      -0.013043814,
      -0.0074874135,
      0.13473365,
      -0.044178527,
      -0.01403677,
      0.00055984996,
      -0.0036781211,
      -0.00014187745,
      -0.010651638,
      0.017360387,
      -0.04366142,
      0.052630052,
      -0.040490936,
      -0.0036669776,
      -0.010526219,
      -0.008558216,
      0.04945971,
      -0.0064618,
      -0.026096895,
      0.03528522,
      0.017515585,
      0.048792165,
      0.012806723,
      0.013030134,
      -0.0066259233,
      0.0021932635,
      -0.01601817,
      0.008459671,
      0.051649842,
      -0.044286244,
      0.050648786,
      -0.021836527,
      0.031767294,
      0.009081719,
      0.039486963,
      -0.031478874,
      -0.005295293,
      0.02604293,
      -0.011712433,
      0.0026289816,
      -0.02771389,
      -0.05880322,
      0.008625529,
      -0.038586393,
      -0.00060886715,
      0.022293508,
      0.01962499,
      -0.039559986,
      0.012422695,
      0.017097618,
      0.0042233826,
      0.016124574,
      0.06400231,
      0.01013828,
      0.0019418234,
      0.0025327099,
      0.06797062,
      -0.01639999,
      0.019147225,
      0.077511005,
      0.04300279,
      -0.0011224708,
      -0.02281973,
      -0.0041384688,
      -0.031457845,
      0.034441024,
      -0.017100282,
      -0.042244416,
      -0.059911303,
      0.020320196,
      0.016682073,
      -0.026526595,
      -0.013109187,
      0.046623614,
      0.03522876,
      -0.07020897,
      -0.0027858827,
      0.0012248874,
      -0.019706104,
      4.449175e-05,
      -0.01676711,
      0.013618695,
      0.016751442,
      0.052337922,
      0.016704846,
      -0.039870158,
      -0.009210376,
      -0.016101042,
      -0.053912774,
      -0.01030838,
      -0.0014928665,
      -0.03861485,
      -0.02639443,
      -0.031236589,
      -0.0367676,
      -0.04256473,
      -0.034367092,
      -0.0015953152,
      -0.039764587,
      0.03163281,
      0.10080723,
      -0.0064988043,
      0.024672346,
      0.039612733,
      -0.051571015,
      0.014494058,
      -0.022822127,
      -0.05607827,
      0.0045406506,
      -0.029270528,
      -0.03174555,
      -0.00014189756,
      -0.010326455,
      0.021104474,
      -0.018238109,
      0.01693558,
      -0.042893708,
      -0.0066386843,
      0.0029949849,
      -0.029824559,
      -0.07980172,
      0.009837255,
      0.042697694,
      0.021210462,
      0.006992767,
      -0.012153643,
      -0.018022908,
      0.06730532,
      0.0097185755,
      -0.010519779,
      -0.03515423,
      0.041359305,
      -0.04106254,
      0.042712715,
      -0.011240963,
      0.019320335,
      0.038198043,
      -0.002018983,
      -0.019421624,
      -0.012232834,
      -0.03197243,
      0.042729717,
      0.0016120019,
      -0.023235232,
      0.019547952,
      0.0061417874,
      -0.0031949624,
      0.029037619,
      0.09187076,
      0.03527133,
      0.03177622,
      -0.03131322,
      -0.057029262,
      0.037499532,
      -0.076599635,
      -0.010948816,
      -0.010056306,
      -0.049771722,
      0.08908218,
      -0.046434876,
      -0.009427074,
      0.08589484,
      -0.020439578,
      0.022801418,
      0.029852122,
      0.009927706,
      -0.046594735,
      -0.019958388,
      0.0062652137,
      -0.00026330803,
      -0.008028819,
      0.064202406,
      0.0025747651,
      0.012016399,
      0.06299626,
      0.010334369,
      -0.0014274095,
      -0.022009145,
      -0.056092076,
      0.02973419,
      0.05777413,
      0.05728468,
      0.056977335,
      -0.029541055,
      -0.009398629,
      -0.004829944,
      -0.038161278,
      0.056747198,
      -0.029851086,
      -0.016171122,
      0.017964598,
      0.039172452,
      0.020577038,
      0.022588894,
      -0.024253663,
      0.028916452
    ]
  }
}
7

@Falco any thoughts on what’s going on here? The errors don’t seem to be stopping.

8

Found out that the Gemini Embeddings API has a hard limit on 10k bytes on the request body, which can be crossed even when staying under the 2048 token limit :facepalm:

This PR will fix it

4 Likes
9

Ok, I updated the discourse plugins from the admin upgrade panel. Unfortunately it’s still throwing errors:

Message (1397 copies reported)

Job exception: Net::HTTPBadResponse


Backtrace

/var/www/discourse/plugins/discourse-ai/lib/inference/gemini_embeddings.rb:17:in `perform!'
/var/www/discourse/plugins/discourse-ai/lib/embeddings/vector_representations/gemini.rb:46:in `vector_from'
/var/www/discourse/plugins/discourse-ai/lib/embeddings/vector_representations/base.rb:178:in `generate_representation_from'
/var/www/discourse/plugins/discourse-ai/app/jobs/scheduled/embeddings_backfill.rb:125:in `block in populate_topic_embeddings'
/var/www/discourse/plugins/discourse-ai/app/jobs/scheduled/embeddings_backfill.rb:122:in `each'
/var/www/discourse/plugins/discourse-ai/app/jobs/scheduled/embeddings_backfill.rb:122:in `populate_topic_embeddings'
/var/www/discourse/plugins/discourse-ai/app/jobs/scheduled/embeddings_backfill.rb:36:in `execute'
/var/www/discourse/app/jobs/base.rb:297:in `block (2 levels) in perform'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rails_multisite-5.0.0/lib/rails_multisite/connection_management.rb:82:in `with_connection'
/var/www/discourse/app/jobs/base.rb:284:in `block in perform'
1 Like
10

This solved the problems on my install, are you sure you updated after the PR was merged? And that the errors are after that change was merged?

11

Hmm, updated it last night from the Admin Upgrade panel. This is what it’s showing as of now:

image
image1117×712 24.3 KB

The PR commit is pointing to aabff87

image
image1320×804 64.9 KB

Your commit is 4 commits after this at a1f1067 but discourse is saying that the Plugin up to date. I’ve even tried restarting the machine but it’s still saying the plugin is up to date.

Do I need change the branch or something? How do I force discourse to install the latest commits to plugin?

image
image957×292 6.85 KB

12

Are you running Discourse at the stable branch by any chance? That will lock the plugin at an old commit. I don’t recommend using the AI plugin with stable, too much moving parts necessitate new core hooks.

3 Likes
13

Moving to tests-passed seems to have upgraded all the plugins and fixed the issue.

I had to move from tests-passed to stable some time ago because of issues I was experiencing with tests-passed which led to downtime. I will say that being on tests-passed does require the admin to be fairly savvy and they can run into problems (like I had with the smtp TLS issues and a few others) and be handy with debugging and getting your hands dirty. It’s not a set it and forget it branch.

However I totally understand why I need to be on tests-passed to use these new AI plugins. Awesome work! Can’t wait to explore it some more.

14

A post was split to a new topic: Encountering Error in Sentiment Backfill Rake Task