### Author:  [@icaria36](https://meta.discourse.org/u/icaria36)
#### Post date: [November 22, 2021, 5:30pm UTC](https://meta.discourse.org/t/js-embed-fails-with-job-exception-invalid-stored-block-lengths-zlib-dataerror/209867/1 "2021-11-22T17:30:57Z")
Hi, we cannot get [JS embed](https://meta.discourse.org/t/embedding-discourse-comments-via-javascript/31963) to work in our setup. I have read all the other similar topics and I have got this embedding to work in a total different project in the past. This one looks like something different…
On the external site, only our logo can be seen followed by “loading discussion…”
On Discourse, no topics are being created, even if the error log shows that the requests are coming.
We have reviewed the URLs. We have also tested adding a static URL (instead of a variable). Nothing.
Discourse error log has this Error:
```
Job exception: invalid stored block lengths (Zlib::DataError)
```
```plaintext
excon-0.88.0/lib/excon/middlewares/decompress.rb:23:in `inflate'
excon-0.88.0/lib/excon/middlewares/decompress.rb:23:in `response_call'
excon-0.88.0/lib/excon/middlewares/base.rb:26:in `response_call'
excon-0.88.0/lib/excon/middlewares/instrumentor.rb:44:in `response_call'
excon-0.88.0/lib/excon/middlewares/base.rb:26:in `response_call'
excon-0.88.0/lib/excon/middlewares/expects.rb:20:in `response_call'
excon-0.88.0/lib/excon/middlewares/response_parser.rb:12:in `response_call'
excon-0.88.0/lib/excon/connection.rb:451:in `response'
excon-0.88.0/lib/excon/connection.rb:282:in `request'
excon-0.88.0/lib/excon.rb:250:in `get'
/var/www/discourse/lib/final_destination.rb:206:in `public_send'
/var/www/discourse/lib/final_destination.rb:206:in `resolve'
/var/www/discourse/app/models/topic_embed.rb:120:in `find_remote'
/var/www/discourse/app/models/topic_embed.rb:192:in `import_remote'
/var/www/discourse/lib/topic_retriever.rb:52:in `fetch_http'
/var/www/discourse/lib/topic_retriever.rb:39:in `perform_retrieve'
/var/www/discourse/lib/topic_retriever.rb:12:in `retrieve'
/var/www/discourse/app/jobs/regular/retrieve_topic.rb:15:in `execute'
/var/www/discourse/app/jobs/base.rb:232:in `block (2 levels) in perform'
rails_multisite-4.0.0/lib/rails_multisite/connection_management.rb:80:in `with_connection'
/var/www/discourse/app/jobs/base.rb:221:in `block in perform'
/var/www/discourse/app/jobs/base.rb:217:in `each'
/var/www/discourse/app/jobs/base.rb:217:in `perform'
sidekiq-6.3.1/lib/sidekiq/processor.rb:196:in `execute_job'
sidekiq-6.3.1/lib/sidekiq/processor.rb:164:in `block (2 levels) in process'
sidekiq-6.3.1/lib/sidekiq/middleware/chain.rb:138:in `block in invoke'
/var/www/discourse/lib/sidekiq/pausable.rb:138:in `call'
sidekiq-6.3.1/lib/sidekiq/middleware/chain.rb:140:in `block in invoke'
sidekiq-6.3.1/lib/sidekiq/middleware/chain.rb:143:in `invoke'
sidekiq-6.3.1/lib/sidekiq/processor.rb:163:in `block in process'
sidekiq-6.3.1/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'
sidekiq-6.3.1/lib/sidekiq/job_retry.rb:112:in `local'
sidekiq-6.3.1/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'
sidekiq-6.3.1/lib/sidekiq.rb:39:in `block in '
sidekiq-6.3.1/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
sidekiq-6.3.1/lib/sidekiq/processor.rb:257:in `stats'
sidekiq-6.3.1/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
sidekiq-6.3.1/lib/sidekiq/job_logger.rb:13:in `call'
sidekiq-6.3.1/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
sidekiq-6.3.1/lib/sidekiq/job_retry.rb:79:in `global'
sidekiq-6.3.1/lib/sidekiq/processor.rb:124:in `block in dispatch'
sidekiq-6.3.1/lib/sidekiq/logger.rb:11:in `with'
sidekiq-6.3.1/lib/sidekiq/job_logger.rb:33:in `prepare'
sidekiq-6.3.1/lib/sidekiq/processor.rb:123:in `dispatch'
sidekiq-6.3.1/lib/sidekiq/processor.rb:162:in `process'
sidekiq-6.3.1/lib/sidekiq/processor.rb:78:in `process_one'
sidekiq-6.3.1/lib/sidekiq/processor.rb:68:in `run'
sidekiq-6.3.1/lib/sidekiq/util.rb:43:in `watchdog'
sidekiq-6.3.1/lib/sidekiq/util.rb:52:in `block in safe_thread'
```
Searching for this error message brings nothing here, neither on… the rest of the Internet?
Any hints?
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [November 22, 2021, 5:40pm UTC](https://meta.discourse.org/t/js-embed-fails-with-job-exception-invalid-stored-block-lengths-zlib-dataerror/209867/2 "2021-11-22T17:40:02Z")
Looks like an error related to the page compression? Can you share a link the to the page you trying to embed?
---
### Author:  [@icaria36](https://meta.discourse.org/u/icaria36)
#### Post date: [November 22, 2021, 6:47pm UTC](https://meta.discourse.org/t/js-embed-fails-with-job-exception-invalid-stored-block-lengths-zlib-dataerror/209867/4 "2021-11-22T18:47:55Z")
Thank you for looking at this!
This is the external page: [Bitwig Preset: Phase-3 | Bitwiggers](https://bitwiggers.com/presets/d1a7c2c7-848d-425c-9058-993317cbcc9c/)
And this is the category where topics should appear: [Bitwiggers.com - Bitwish](https://bitwish.top/c/bitwiggers/9)
(We have removed the template after the failed tests.)
Does this comment help? (this is out of my tech league):
> The responses are successful 200s, and based on that discussion it seems like it’s failing to handle that response while scraping Bitwiggers, and it’s specifically failing when trying to handle the compression.
>
> Based on `inflate` , it seems like it negotiated to use the `deflate` compression scheme, whereas I think the preferred would be `gzip` . Firefox’s requests indicate that it would handle both, but the server decides on `gzip` in that case.
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [November 22, 2021, 8:41pm UTC](https://meta.discourse.org/t/js-embed-fails-with-job-exception-invalid-stored-block-lengths-zlib-dataerror/209867/5 "2021-11-22T20:41:23Z")
Looks like if I send a request to your webserver as
```plaintext
curl 'https://bitwiggers.com/presets/d1a7c2c7-848d-425c-9058-993317cbcc9c/' -H 'accept-encoding: deflate, gzip' -vv
```
I get back a `deflate` compressed response, where most webservers when given the two options would prefer using `gzip`. That’s weird.
I see that I can easily reproduce this error with
```ruby
require 'excon'
response = Excon.get('https://bitwiggers.com/presets/d1a7c2c7-848d-425c-9058-993317cbcc9c/',
headers: {'accept-encoding' => 'deflate, gzip'})
# fails, default in excon
Zlib::Inflate.new(-Zlib::MAX_WBITS).inflate(response.body)
#works
Zlib::Inflate.new(0).inflate(response.body)
```
This code is over 8 years old in Excon, so I’d guess your webserver is at fault here, but I reported it upstream anyway
[https://github.com/excon/excon/issues/768](https://github.com/excon/excon/issues/768)
---
### Author:  [@icaria36](https://meta.discourse.org/u/icaria36)
#### Post date: [November 22, 2021, 9:17pm UTC](https://meta.discourse.org/t/js-embed-fails-with-job-exception-invalid-stored-block-lengths-zlib-dataerror/209867/8 "2021-11-22T21:17:11Z")
Interesting, and thank you for filing the bug. Subscribed.
Trying to summarize our thoughts:
We have no control over that handling, but we don’t think it’s the webserver. What about the construction of the headers instead? If passed in as `gzip, deflate` , then the response should be gzipped. According to HTTP Specifications, the client should present the options, weighted or not, in order according to their preference. so if the request is preferring deflate over gzip, then gets a deflate. Isn’t then the problem in the code/request, because it received an encoding that it asked for and failed to handle it?
Maybe the problem is related to this?
>
There is a “merge” and what seems to be an expectation for gzip?
The admin of the other server is looking at how can gzip be forced on their end. Still, it would be interesting to know your thoughts about this.
(For reference, the server is AWS API Gateway; not something we have control over, aside from enabling/disabling compression.)
---
### Author:  [@icaria36](https://meta.discourse.org/u/icaria36)
#### Post date: [November 28, 2021, 2:28pm UTC](https://meta.discourse.org/t/js-embed-fails-with-job-exception-invalid-stored-block-lengths-zlib-dataerror/209867/9 "2021-11-28T14:28:01Z")
> [@Falco](#):
>
> I reported it upstream anyway
The developer has responded: [Issue · GitHub](https://github.com/excon/excon/issues/768#issuecomment-981085873)
---
### Author:  [@icaria36](https://meta.discourse.org/u/icaria36)
#### Post date: [December 2, 2021, 9:58pm UTC](https://meta.discourse.org/t/js-embed-fails-with-job-exception-invalid-stored-block-lengths-zlib-dataerror/209867/10 "2021-12-02T21:58:02Z")
Could some help translating [the response from the Excon developer](https://github.com/excon/excon/issues/768#issuecomment-981085873)? I am not sure where is the problem and who could fix it.
I also wonder whether we could apply a local patch to our Discourse to circumvent this gzip/deflate problem. Local patches are painful, but it is even more painful to have an integration of two sites with hundreds of pages to be embedded with Discourse discussions pending because of this problem.
---
### Author:  [@icaria36](https://meta.discourse.org/u/icaria36)
#### Post date: [December 5, 2021, 7:24pm UTC](https://meta.discourse.org/t/js-embed-fails-with-job-exception-invalid-stored-block-lengths-zlib-dataerror/209867/11 "2021-12-05T19:24:09Z")
The Excon maintainer has [committed](https://github.com/excon/excon/issues/768#issuecomment-986233519) a patch that in theory solves this problem! 🎉 A new Excon release should come soon, he says.
---
### Author:  [@icaria36](https://meta.discourse.org/u/icaria36)
#### Post date: [December 8, 2021, 8:58am UTC](https://meta.discourse.org/t/js-embed-fails-with-job-exception-invalid-stored-block-lengths-zlib-dataerror/209867/12 "2021-12-08T08:58:02Z")
And now [the Excon maintainer has published a new version](https://github.com/excon/excon/issues/768#issuecomment-986233519) of the library including this patch. 🎉
Does this mean that we can update our instance right away, or is Discourse using their own staged repository for these libraries? Remembering the output of Discourse upgrades I believe that they are being pulled directly from upstream repositories, but I’d rather ask before touching our backend. 🙂
---
### Author:  [@icaria36](https://meta.discourse.org/u/icaria36)
#### Post date: [December 10, 2021, 9:56pm UTC](https://meta.discourse.org/t/js-embed-fails-with-job-exception-invalid-stored-block-lengths-zlib-dataerror/209867/13 "2021-12-10T21:56:00Z")
I decided to ask Discourse directly 🙂 by upgrading and checking the logs. After seeing this
> Installing excon 0.89.0
> www/discourse/vendor/bundle/ruby/2.7.0/specifications/excon-0.89.0.gemspec
we have tried again and NOW EMBEDDING WORKS!
Thank you so much @Falco for your report upstream. You were absolutely right, and I added some noise that perhaps wasn’t that useful. My apologies, and thank you again.
---
### Author:  [@system](https://meta.discourse.org/u/system)
#### Post date: [January 9, 2022, 9:56pm UTC](https://meta.discourse.org/t/js-embed-fails-with-job-exception-invalid-stored-block-lengths-zlib-dataerror/209867/14 "2022-01-09T21:56:47Z")
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.