Add options to compress callback request payload

[datenzauberai]

This implements an option to compress the payload of a callback's request (see #3924).

This would be useful when transmitting large amounts of data from the client-side to the server-side (server to client is already covered by standard HTTP compression via Content-Encoding).

The basic idea is that the server side callbacks can be marked for payload compression with two parameters compress_payload: bool and compress_threshold: int that indicate if the request body should be compressed and the minimum size of body before compressing.

@app.callback(
    Output("my-plot", "figure"),
    Input("refresh-button", "n_clicks"),
    State("my-grid", "rowData"),
    State("my-grid", "columnDefs"),
    State("my-store", "data"),
    compress_payload=True,
    compress_threshold=2_500,
)
def update_plot(n, gridRowData, gridColumnDefs, storeData):

On the client side the JSON of the body is constructed as before. When the callback is marked for request compression and the request body exceeds the compression threshold, the body is compressed and send as binary with the header Content-Encoding=gzip.

On the server side the request is checked for the content encoding header and the request body is decompressed when necessary.

Advantage of this solution is that it does not depend on compressing individual properties and can be used without compromising client-side callbacks to be able to access properties.

Downside is that a new dependency has to be introduced to actually compress the payload in the browser (I opted for https://github.com/101arrowz/fflate).

I would expect this change to be able to really speed-up application performance for use-cases where loads of data are transmitted and would love to get some feedback on this. I did some experiments and observed roundtrip response time improvements up to 10x. Some numbers showing the average speed-up ratio of the roundtrip response time depending for different connections and the payload sizes.

Compression almost never hurts the roundtrip response times which is why I would go for a default compress_threshold of around 5 KB - 50 KB (but keep compress_payload=False per default).

Naming of the parameters could be adjusted if we want to emphasise that they only apply to the request part of the callback and not the response. Might also be an option to have a single parameter that takes a dict.

This seems to work fine across all backends (flask, fastapi, and quart).

I first tried sending the compressed payload as Base64-encoded data inside JSON, but sending it as raw binary is significantly more efficient. It is also less proprietary, avoids adding a base64 package to the dash renderer, and allows a reverse proxy to decompress the payload transparently (which would reset/remove the content encoding header).

Contributor Checklist

• I have broken down my PR scope into the following TODO tasks
  • Added parameters compress_payload and compress_threshold to callbacks
  • Implemented client-side compression and server-side decompression of the callback payload
• I have run the tests locally and they passed. (refer to testing section in contributing)
• I have added tests, or extended existing tests, to cover any new features or bugs fixed in this PR

optionals

• I have added entry in the CHANGELOG.md
• If this PR needs a follow-up in dash docs, community thread, I have mentioned the relevant URLS as follows
  • this GitHub #PR number updates the dash docs
  • here is the show and tell thread in Plotly Dash community

[Aaron-Wrote-This]

Doesn't dash already support this via the compress option? https://dash.plotly.com/reference

Maybe adding support for fast api's compression directly in dash could be helpful too: https://fastapi.tiangolo.com/advanced/middleware/#gzipmiddleware
No new library's needed.

And ideally you don't really want dash running compression on the main thread in most cases. Having the server do it instead is generally better/faster/more flexible, such as nginx https://docs.nginx.com/nginx/admin-guide/web-server/compression/

[datenzauberai]

Doesn't dash already support this via the compress option? https://dash.plotly.com/reference

[…]

And ideally you don't really want dash running compression on the main thread in most cases. Having the server do it instead is generally better/faster/more flexible, such as nginx https://docs.nginx.com/nginx/admin-guide/web-server/compression/

This was also my first thought when I looked into this! Unfortunately, all these standard compression mechanism will only work for the HTTP response from the server to the client downloading the data. This PR would address the other direction when uploading the data which is not covered by the standard means. I enhanced the description to make this a little more clear.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
2.6% Duplication on New Code

See analysis details on SonarQube Cloud