Update django-storages to 1.11
Created by: pyup-bot
This PR updates django-storages from 1.6.5 to 1.11.
Changelog
1.11
*****************
General
-------
- Test against Python 3.9 (`964`_)
S3
--
- Fix ``ValueError: I/O operation on closed file`` when calling ``collectstatic`` (`382`_, `955`_)
- Calculate ``S3Boto3StorageFile.buffer_size`` (via setting ``AWS_S3_FILE_BUFFER_SIZE``)
at run-time rather than import-time. (`930`_)
- Fix writing ``bytearray`` content (`958`_, `965`_)
Google Cloud
------------
- Add setting ``GS_QUERYSTRING_AUTH`` to avoid signing URLs. This is useful for buckets with a
policy of Uniform public read (`952`_)
Azure
-----
- Add ``AZURE_OBJECT_PARAMETERS`` and overridable ``AzureStorage.get_object_parameters`` to customize
``ContentSettings`` parameters for all keys and per-key respectively. (`898`_)
.. _382: https://github.com/jschneier/django-storages/issues/382
.. _955: https://github.com/jschneier/django-storages/pull/955
.. _930: https://github.com/jschneier/django-storages/pull/930
.. _952: https://github.com/jschneier/django-storages/pull/952
.. _898: https://github.com/jschneier/django-storages/pull/898
.. _964: https://github.com/jschneier/django-storages/pull/964
.. _958: https://github.com/jschneier/django-storages/issues/958
.. _965: https://github.com/jschneier/django-storages/pull/9651.10.1
*******************
S3
--
- Restore ``AWS_DEFAULT_ACL`` handling. This setting is ignored if ``ACL`` is set in
``AWS_S3_OBJECT_PARAMETERS`` (`934`_)
SFTP
----
- Fix using ``SFTP_STORAGE_HOST`` (`926`_)
.. _926: https://github.com/jschneier/django-storages/pull/926
.. _934: https://github.com/jschneier/django-storages/pull/9341.10
*****************
General
-------
- **Breaking**: Removed support for end-of-life Python 2.7 and 3.4 (`709`_)
- **Breaking**: Removed support for end-of-life Django 1.11 (`891`_)
- Add support for Django 3.1 (`916`_)
- Introduce a new ``BaseStorage`` class with a ``get_default_settings`` method and use
it in ``S3Boto3Storage``, ``AzureStorage``, ``GoogleCloudStorage``, and ``SFTPStorage``. These backends
now calculate their settings when instantiated, not imported. (`524`_, `852`_)
S3
--
- **Breaking**: Automatic bucket creation has been removed. Doing so encourages using overly broad credentials.
As a result, support for the corresponding ``AWS_BUCKET_ACL`` and ``AWS_AUTO_CREATE_BUCKET`` settings have been removed. (`636`_)
- **Breaking**: Support for the undocumented setting ``AWS_PRELOAD_METADATA`` has been removed (`636`_)
- **Breaking**: The constructor kwarg ``acl`` is no longer accepted. Instead, use the ``ACL`` key in setting ``AWS_S3_OBJECT_PARAMETERS``
(`636`_)
- **Breaking**: The constructor kwarg ``bucket`` is no longer accepted. Instead, use ``bucket_name`` or the ``AWS_STORAGE_BUCKET_NAME``
setting (`636`_)
- **Breaking**: Support for setting ``AWS_REDUCED_REDUNDANCY`` has been removed. Replace with ``StorageClass=REDUCED_REDUNDANCY``
in ``AWS_S3_OBJECT_PARAMETERS`` (`636`_)
- **Breaking**: Support for setting ``AWS_S3_ENCRYPTION`` has been removed. Replace with ``ServerSideEncryption=AES256`` in ``AWS_S3_OBJECT_PARAMETERS`` (`636`_)
- **Breaking**: Support for setting ``AWS_DEFAULT_ACL`` has been removed. Replace with ``ACL`` in ``AWS_S3_OBJECT_PARAMETERS`` (`636`_)
- Add ``http_method`` parameter to ``.url`` method (`854`_)
- Add support for signing Cloudfront URLs to the ``.url`` method. You must set ``AWS_CLOUDFRONT_KEY``,
``AWS_CLOUDFRONT_KEY_ID`` and install either `cryptography`_ or `rsa`_ (`456`_, `587`_). See the docs for more info.
URLs will only be signed if ``AWS_QUERYSTRING_AUTH`` is set to ``True`` (`885`_)
Google Cloud
------------
- **Breaking**: Automatic bucket creation has been removed. Doing so encourages using overly broad credentials.
As a result, support for the corresponding ``GS_AUTO_CREATE_BUCKET`` and ``GS_AUTO_CREATE_ACL`` settings have been removed. (`894`_)
Dropbox
-------
- Add ``DROPBOX_WRITE_MODE`` setting to control e.g. overwriting behavior. Check the docs
for more info (`873`_, `138`_)
SFTP
----
- Remove exception swallowing during ssh connection (`835`_, `838`_)
FTP
---
- Add ``FTP_STORAGE_ENCODING`` setting to set the filesystem encoding (`803`_)
- Support multiple nested paths for files (`886`_)
.. _cryptography: https://cryptography.io
.. _rsa: https://stuvel.eu/rsa
.. _885: https://github.com/jschneier/django-storages/pull/885
.. _894: https://github.com/jschneier/django-storages/pull/894
.. _636: https://github.com/jschneier/django-storages/pull/636
.. _709: https://github.com/jschneier/django-storages/pull/709
.. _891: https://github.com/jschneier/django-storages/pull/891
.. _916: https://github.com/jschneier/django-storages/pull/916
.. _852: https://github.com/jschneier/django-storages/pull/852
.. _873: https://github.com/jschneier/django-storages/pull/873
.. _854: https://github.com/jschneier/django-storages/pull/854
.. _138: https://github.com/jschneier/django-storages/issues/138
.. _524: https://github.com/jschneier/django-storages/pull/524
.. _835: https://github.com/jschneier/django-storages/issues/835
.. _838: https://github.com/jschneier/django-storages/pull/838
.. _803: https://github.com/jschneier/django-storages/pull/803
.. _456: https://github.com/jschneier/django-storages/issues/456
.. _587: https://github.com/jschneier/django-storages/pull/587
.. _886: https://github.com/jschneier/django-storages/pull/8861.9.1
******************
S3
--
- Fix reading files with ``S3Boto3StorageFile`` (`831`_, `833`_)
.. _831: https://github.com/jschneier/django-storages/issues/831
.. _833: https://github.com/jschneier/django-storages/pull/8331.9
****************
General
-------
- **Breaking**: The long deprecated S3 backend based on ``boto`` has been removed. (`825`_)
- Test against and support Python 3.8 (`810`_)
S3
--
- **Deprecated**: Automatic bucket creation will be removed in version 1.10 (`826`_)
- **Deprecated**: The undocumented ``AWS_PRELOAD_METADATA`` and associated functionality will
be removed in version 1.10 (`829`_)
- **Deprecated**: Support for ``AWS_REDUCED_REDUNDANCY`` will be removed in version 1.10
Replace with ``StorageClass=REDUCED_REDUNDANCY`` in ``AWS_S3_OBJECT_PARAMETERS`` (`829`_)
- **Deprecated**: Support for ``AWS_S3_ENCRYPTION`` will be removed in version 1.10 (`829`_)
Replace with ``ServerSideEncryption=AES256`` in ``AWS_S3_OBJECT_PARAMETERS``
- A custom ``ContentEncoding`` is no longer overwritten automatically (note that specifying
one will disable automatic ``gzip``) (`391`_, `828`_).
- Add ``S3Boto3Storage.get_object_parameters``, an overridable method for customizing
upload parameters on a per-object basis (`819`_, `828`_)
- Opening and closing a file in `w` mode without writing anything will now create an empty file
in S3, this mimics the builtin ``open`` and Django's own ``FileSystemStorage`` (`435`_, `816`_)
- Fix reading a file in text mode (`404`_, `827`_)
Google Cloud
------------
- **Deprecated**: Automatic bucket creation will be removed in version 1.10 (`826`_)
Dropbox
-------
- Fix crash on ``DropBoxStorage.listdir`` (`762`_)
- Settings can now additionally be specified at the class level to ease subclassing (`745`_)
Libcloud
--------
- Add support for Backblaze B2 to ``LibCloudStorage.url`` (`807`_)
FTP
---
- Fix creating multiple intermediary directories on Windows (`823`_, `824`_)
.. _825: https://github.com/jschneier/django-storages/pull/825
.. _826: https://github.com/jschneier/django-storages/pull/826
.. _829: https://github.com/jschneier/django-storages/pull/829
.. _391: https://github.com/jschneier/django-storages/issues/391
.. _828: https://github.com/jschneier/django-storages/pull/828
.. _819: https://github.com/jschneier/django-storages/issues/819
.. _810: https://github.com/jschneier/django-storages/pull/810
.. _435: https://github.com/jschneier/django-storages/issues/435
.. _816: https://github.com/jschneier/django-storages/pull/816
.. _404: https://github.com/jschneier/django-storages/issues/404
.. _827: https://github.com/jschneier/django-storages/pull/827
.. _762: https://github.com/jschneier/django-storages/pull/762
.. _745: https://github.com/jschneier/django-storages/pull/745
.. _807: https://github.com/jschneier/django-storages/pull/807
.. _823: https://github.com/jschneier/django-storages/issues/823
.. _824: https://github.com/jschneier/django-storages/pull/8241.8
****************
General
-------
- Add support for Django 3.0 (`759`_)
- Update license identifier to unambiguous ``BSD-3-Clause``
S3
--
- Include error message raised when missing library is imported (`776`_, `793`_)
Google
------
- **Breaking** The minimum supported version of ``google-cloud-storage`` is now ``1.15.0`` which enables...
- Add setting ``GS_CUSTOM_ENDPOINT`` to allow usage of custom domains (`775`_, `648`_)
Azure
-----
- Fix extra installation by pinning version to < 12 (`785`_)
- Add support for setting ``AZURE_CACHE_CONTROL`` header (`780`_, `674`_)
.. _759: https://github.com/jschneier/django-storages/pull/759
.. _776: https://github.com/jschneier/django-storages/issues/776
.. _793: https://github.com/jschneier/django-storages/pull/793
.. _775: https://github.com/jschneier/django-storages/issues/775
.. _648: https://github.com/jschneier/django-storages/pull/648
.. _785: https://github.com/jschneier/django-storages/pull/785
.. _780: https://github.com/jschneier/django-storages/pull/780
.. _674: https://github.com/jschneier/django-storages/issues/6741.7.2
******************
S3
--
- Avoid misleading ``AWS_DEFAULT_ACL`` warning for insecure ``default_acl`` when
overridden as a class variable (`591`_)
- Propagate file deletion to cache when ``preload_metadata`` is ``True``,
(not the default) (`743`_, `749`_)
- Fix exception raised on closed file (common if using ``ManifestFilesMixin`` or
``collectstatic``. (`382`_, `754`_)
Azure
-----
- Pare down the required packages in ``extra_requires`` when installing the ``azure`` extra to only
``azure-storage-blob`` (`680`_, `684`_)
- Fix compatability with ``generate_blob_shared_access_signature`` updated signature (`705`_, `723`_)
- Fetching a file now uses the configured timeout rather than hardcoding one (`727`_)
- Add support for configuring all blobservice options: ``AZURE_EMULATED_MODE``, ``AZURE_ENDPOINT_SUFFIX``,
``AZURE_CUSTOM_DOMAIN``, ``AZURE_CONNECTION_STRING``, ``AZURE_CUSTOM_CONNECTION_STRING``,
``AZURE_TOKEN_CREDENTIAL``. See the docs for more info. Huge thanks once again to nitely. (`750`_)
- Fix filename handling to not strip special characters (`609`_, `752`_)
Google Cloud
------------
- Set the file acl in the same call that uploads it (`698`_)
- Reduce the number of queries and required permissions when ``GS_AUTO_CREATE_BUCKET`` is
``False`` (the default) (`412`_, `718`_)
- Set the ``predefined_acl`` when creating a ``GoogleCloudFile`` using ``.write``
(`640`_, `756`_)
- Add ``GS_BLOB_CHUNK_SIZE`` setting to enable efficient uploading of large files (`757`_)
Dropbox
-------
- Complete migration to v2 api with file fetching and metadata fixes (`724`_)
- Add ``DROPBOX_TIMEOUT`` to configure client timeout defaulting to 100 seconds
to match the underlying sdk. (`419`_, `747`_)
SFTP
----
- Fix reopening a file (`746`_)
.. _591: https://github.com/jschneier/django-storages/pull/591
.. _680: https://github.com/jschneier/django-storages/issues/680
.. _684: https://github.com/jschneier/django-storages/pull/684
.. _698: https://github.com/jschneier/django-storages/pull/698
.. _705: https://github.com/jschneier/django-storages/issues/705
.. _723: https://github.com/jschneier/django-storages/pull/723
.. _727: https://github.com/jschneier/django-storages/pull/727
.. _746: https://github.com/jschneier/django-storages/pull/746
.. _724: https://github.com/jschneier/django-storages/pull/724
.. _412: https://github.com/jschneier/django-storages/pull/412
.. _718: https://github.com/jschneier/django-storages/pull/718
.. _743: https://github.com/jschneier/django-storages/issues/743
.. _749: https://github.com/jschneier/django-storages/pull/749
.. _750: https://github.com/jschneier/django-storages/pull/750
.. _609: https://github.com/jschneier/django-storages/issues/609
.. _752: https://github.com/jschneier/django-storages/pull/752
.. _382: https://github.com/jschneier/django-storages/issues/382
.. _754: https://github.com/jschneier/django-storages/pull/754
.. _419: https://github.com/jschneier/django-storages/issues/419
.. _747: https://github.com/jschneier/django-storages/pull/747
.. _640: https://github.com/jschneier/django-storages/issues/640
.. _756: https://github.com/jschneier/django-storages/pull/756
.. _757: https://github.com/jschneier/django-storages/pull/7571.7.1
******************
- Fix off-by-1 error in ``get_available_name`` whenever ``file_overwrite`` or ``overwrite_files`` is ``True`` (`588`_, `589`_)
- Change ``S3Boto3Storage.listdir()`` to use ``list_objects`` instead of ``list_objects_v2`` to restore
compatability with services implementing the S3 protocol that do not yet support the new method (`586`_, `590`_)
.. _588: https://github.com/jschneier/django-storages/issues/588
.. _589: https://github.com/jschneier/django-storages/pull/589
.. _586: https://github.com/jschneier/django-storages/issues/586
.. _590: https://github.com/jschneier/django-storages/pull/5901.7
****************
**Security**
- The ``S3BotoStorage`` and ``S3Boto3Storage`` backends have an insecure
default ACL of ``public-read``. It is recommended that all current users audit their bucket
permissions. Support has been added for setting ``AWS_DEFAULT_ACL = None`` and ``AWS_BUCKET_ACL =
None`` which causes all created files to inherit the bucket's ACL (and created buckets to inherit the
Amazon account's default ACL). This will become the default in version 1.10 (for ``S3Boto3Storage`` only
since ``S3BotoStorage`` will be removed in version 1.9, see below). Additionally, a warning is now
raised if ``AWS_DEFAULT_ACL`` or ``AWS_BUCKET_ACL`` is not explicitly set. (`381`_, `535`_, `579`_)
**Breaking**
- The ``AzureStorage`` backend and documentation has been completely rewritten. It now
depends on ``azure`` and ``azure-storage-blob`` and is *vastly* improved. Big thanks to nitely and all
other contributors along the way (`565`_)
- The ``.url()`` method of ``GoogleCloudStorage`` has been completely reworked. Many use
cases should require no changes and will experience a massive speedup. The ``.url()`` method no longer hits
the network for public urls and generates signed urls (with a default of 1-day expiration, configurable
via ``GS_EXPIRATION``) for non-public buckets. Check out the docs for more information. (`570`_)
- Various backends will now raise ``ImproperlyConfigured`` at runtime if their
location (``GS_LOCATION``, ``AWS_LOCATION``) begins with a leading ``/`` rather than silently
stripping it. Verify yours does not. (`520`_)
- The long deprecated ``GSBotoStorage`` backend is removed. (`518`_)
**Deprecation**
- The insecure default of ``public-read`` for ``AWS_DEFAULT_ACL`` and
``AWS_BUCKET_ACL`` in ``S3Boto3Storage`` will change to inherit the bucket's setting in version 1.10 (`579`_)
- The legacy ``S3BotoBackend`` is deprecated and will be removed in version 1.9.
It is strongly recommended to move to the ``S3Boto3Storage`` backend for performance,
stability and bugfix reasons. See the `boto migration docs`_ for step-by-step guidelines. (`578`_, `584`_)
- The long aliased arguments to ``S3Boto3Storage`` of ``acl`` and ``bucket`` are
deprecated in favor of ``bucket_name`` and ``default_acl`` (`516`_)
- The minimum required version of ``boto3`` will be increasing to ``1.4.4`` in
the next major version of ``django-storages``. (`583`_)
**Features**
- Add support for a file to inherit its bucket's ACL by setting ``AWS_DEFAULT_ACL = None`` (`535`_)
- Add ``GS_CACHE_CONTROL`` setting for ``GoogleCloudStorage`` backend (`411`_, `505`_)
- Add documentation around using django-storages with Digital Ocean Spaces (`521`_)
- Add support for Django 2.1 and Python 3.7 (`530`_)
- Make ``S3Boto3Storage`` pickleable (`551`_)
- Add automatic reconnection to ``SFTPStorage`` (`563`_, `564`_)
- Unconditionally set the security token in the boto backends (`b13efd`_)
- Improve efficiency of ``.listdir`` on ``S3Boto3Storage`` (`352`_)
- Add ``AWS_S3_VERIFY`` to support custom certificates and disabling certificate verification
to ``S3Boto3Storage`` (`486`_, `580`_)
- Add ``AWS_S3_PROXIES`` setting to ``S3Boto3Storage`` (`583`_)
- Add a snazzy new logo. Big thanks to reallinfo
**Bugfixes**
- Reset file read offset before passing to ``GoogleCloudStorage`` and ``AzureStorage`` (`481`_, `581`_, `582`_)
- Fix various issues with multipart uploads in the S3 backends
(`169`_, `160`_, `364`_, `449`_, `504`_, `506`_, `546`_)
- Fix ``S3Boto3Storage`` to stream down large files (also disallow `r+w` mode) (`383`_, `548`_)
- Fix ``SFTPStorageFile`` to align with the core ``File`` abstraction (`487`_, `568`_)
- Catch ``IOError`` in ``SFTPStorage.delete`` (`568`_)
- ``AzureStorage``, ``GoogleCloudStorage``, ``S3Boto3Storage`` and ``S3BotoStorage`` now
respect ``max_length`` when ``file_overwrite = True`` (`513`_, `554`_)
- The S3 backends now consistently use ``compresslevel=9`` (the Python stdlib default)
for gzipped content (`572`_, `576`_)
- Improve error message of ``S3Boto3Storage`` during an unexpected exception when automatically
creating a bucket (`574`_, `577`_)
.. _381: https://github.com/jschneier/django-storages/issues/381
.. _535: https://github.com/jschneier/django-storages/pull/535
.. _579: https://github.com/jschneier/django-storages/pull/579
.. _565: https://github.com/jschneier/django-storages/pull/565
.. _520: https://github.com/jschneier/django-storages/pull/520
.. _518: https://github.com/jschneier/django-storages/pull/518
.. _516: https://github.com/jschneier/django-storages/pull/516
.. _481: https://github.com/jschneier/django-storages/pull/481
.. _581: https://github.com/jschneier/django-storages/pull/581
.. _582: https://github.com/jschneier/django-storages/pull/582
.. _411: https://github.com/jschneier/django-storages/issues/411
.. _505: https://github.com/jschneier/django-storages/pull/505
.. _521: https://github.com/jschneier/django-storages/pull/521
.. _169: https://github.com/jschneier/django-storages/pull/169
.. _160: https://github.com/jschneier/django-storages/issues/160
.. _364: https://github.com/jschneier/django-storages/pull/364
.. _449: https://github.com/jschneier/django-storages/issues/449
.. _504: https://github.com/jschneier/django-storages/pull/504
.. _530: https://github.com/jschneier/django-storages/pull/530
.. _506: https://github.com/jschneier/django-storages/pull/506
.. _546: https://github.com/jschneier/django-storages/pull/546
.. _383: https://github.com/jschneier/django-storages/issues/383
.. _548: https://github.com/jschneier/django-storages/pull/548
.. _b13efd: https://github.com/jschneier/django-storages/commit/b13efd92b3bf3e9967b8e7819224bfcf9abb977e
.. _551: https://github.com/jschneier/django-storages/pull/551
.. _563: https://github.com/jschneier/django-storages/issues/563
.. _564: https://github.com/jschneier/django-storages/pull/564
.. _487: https://github.com/jschneier/django-storages/issues/487
.. _568: https://github.com/jschneier/django-storages/pull/568
.. _513: https://github.com/jschneier/django-storages/issues/513
.. _554: https://github.com/jschneier/django-storages/pull/554
.. _570: https://github.com/jschneier/django-storages/pull/570
.. _572: https://github.com/jschneier/django-storages/issues/572
.. _576: https://github.com/jschneier/django-storages/pull/576
.. _352: https://github.com/jschneier/django-storages/pull/352
.. _574: https://github.com/jschneier/django-storages/issues/574
.. _577: https://github.com/jschneier/django-storages/pull/577
.. _486: https://github.com/jschneier/django-storages/pull/486
.. _580: https://github.com/jschneier/django-storages/pull/580
.. _583: https://github.com/jschneier/django-storages/pull/583
.. _boto migration docs: https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.htmlmigrating-boto-to-boto3
.. _578: https://github.com/jschneier/django-storages/pull/578
.. _584: https://github.com/jschneier/django-storages/pull/5841.6.6
******************
* You can now specify the backend you are using to install the necessary dependencies using
``extra_requires``. For example ``pip install django-storages[boto3]`` (`417`_)
* Add additional content-type detection fallbacks (`406`_, `407`_)
* Add ``GS_LOCATION`` setting to specify subdirectory for ``GoogleCloudStorage`` (`355`_)
* Add support for uploading large files to ``DropBoxStorage``, fix saving files (`379`_, `378`_, `301`_)
* Drop support for Django 1.8 and Django 1.10 (and hence Python 3.3) (`438`_)
* Implement ``get_created_time`` for ``GoogleCloudStorage`` (`464`_)
.. _417: https://github.com/jschneier/django-storages/pull/417
.. _407: https://github.com/jschneier/django-storages/pull/407
.. _406: https://github.com/jschneier/django-storages/issues/406
.. _355: https://github.com/jschneier/django-storages/pull/355
.. _379: https://github.com/jschneier/django-storages/pull/379
.. _378: https://github.com/jschneier/django-storages/issues/378
.. _301: https://github.com/jschneier/django-storages/issues/301
.. _438: https://github.com/jschneier/django-storages/issues/438
.. _464: https://github.com/jschneier/django-storages/pull/464