Improved caching for urls
- When there is no file at the destination yet, we have no modification time for the `If-Modified-Since`-Header. In this case trust the cache to make the right decision to either serve a cached version or to refresh from origin. This should help with mass-deployment scenarios where you want to use a local cache to relieve your uplink.
- If you don't trust the cache to make the right decision you can still force it to refresh by providing the `force: yes` option.
(cherry picked from commit c05bad9f74
)
This commit is contained in:
parent
503537eb25
commit
2c572ba786
1 changed files with 6 additions and 5 deletions
|
@ -853,13 +853,14 @@ def open_url(url, data=None, headers=None, method=None, use_proxy=True,
|
|||
if http_agent:
|
||||
request.add_header('User-agent', http_agent)
|
||||
|
||||
# if we're ok with getting a 304, set the timestamp in the
|
||||
# header, otherwise make sure we don't get a cached copy
|
||||
if last_mod_time and not force:
|
||||
# Cache control
|
||||
# Either we directly force a cache refresh
|
||||
if force:
|
||||
request.add_header('cache-control', 'no-cache')
|
||||
# or we do it if the original is more recent than our copy
|
||||
elif last_mod_time:
|
||||
tstamp = last_mod_time.strftime('%a, %d %b %Y %H:%M:%S +0000')
|
||||
request.add_header('If-Modified-Since', tstamp)
|
||||
else:
|
||||
request.add_header('cache-control', 'no-cache')
|
||||
|
||||
# user defined headers now, which may override things we've set above
|
||||
if headers:
|
||||
|
|
Loading…
Reference in a new issue