Commit graph

4864 commits

Author SHA1 Message Date
Harshavardhana b6ca39ea48 Support migrating inconsistent bucket policies (#5855)
Previously we used allow bucket policies without
`Version` field to be set to any given value, but
this behavior is inconsistent with AWS S3.

PR #5790 addressed this by making bucket policies
stricter and cleaner, but this causes a breaking
change causing any existing policies perhaps without
`Version` field or the field to be empty to fail upon
server startup.

This PR brings a code to migrate under these scenarios
as a one time operation.
2018-04-27 15:02:54 -07:00
Elson Rodriguez 1bd7eb979c Add example on how to deploy minio without a backing store. (#5810)
Many on-premise clusters do not have a PV abstraction, this example 
shows a way to deploy minio effectively in such environments.
2018-04-27 20:58:23 +05:30
poornas 0dc3d7ac18 fix error checks when cache is offline/missing. (#5850) 2018-04-26 10:39:05 +05:30
Krishna Srinivas 9aace6d36d Continue healing other objects even if objects without quorum exist (#5851)
fixes #5815
2018-04-25 11:56:39 -07:00
Krishna Srinivas 6831177394 Do not log errFileNotFound error (#5853) 2018-04-25 11:46:49 -07:00
kannappanr fe126de98b
Regenerate fs.json if it is corrupted in FS mode (#5778)
Also return a default e-tag for pre-existing objects.
Fixes #5712
2018-04-24 17:36:43 -07:00
Bala FA 0d52126023 Enhance policy handling to support SSE and WORM (#5790)
- remove old bucket policy handling
- add new policy handling
- add new policy handling unit tests

This patch brings support to bucket policy to have more control not
limiting to anonymous.  Bucket owner controls to allow/deny any rest
API.

For example server side encryption can be controlled by allowing
PUT/GET objects with encryptions including bucket owner.
2018-04-24 15:53:30 -07:00
Andreas Auernhammer 21a3c0f482 disable elliptic curves P-384 and P-521 for TLS. (#5845)
This change disables the non-constant-time implementations of P-384 and P-521.
As a consequence a client using just these curves cannot connect to the server.
This should be no real issues because (all) clients at least support P-256.

Further this change also rejects ECDSA private keys of P-384 and P-521.
While non-constant-time implementations for the ECDHE exchange don't expose an
obvious vulnerability, using P-384 or P-521 keys for the ECDSA signature may allow
pratical timing attacks.

Fixes #5844
2018-04-24 15:47:30 -07:00
Benjamin c733fe87ce change MAINTAINER to LABEL (#5843) 2018-04-24 09:57:48 -07:00
Harshavardhana ccdb7bc286 Fix s3 compatibility fixes for getBucketLocation,headBucket,deleteBucket (#5842)
- getBucketLocation
- headBucket
- deleteBucket

Should return 404 or NoSuchBucket even for invalid bucket names, invalid
bucket names are only validated during MakeBucket operation
2018-04-24 08:57:33 +05:30
Harshavardhana 954142a98f Cleanup and make a safer code (#5794) 2018-04-21 20:51:53 -07:00
Bala FA 76cc65531c use package name correctly (#5827) 2018-04-21 19:23:54 -07:00
ebozduman f16bfda2f2 Remove panic() and handle it appropriately (#5807)
This is an effort to remove panic from the source. 
Add a new call called CriticialIf, that calls LogIf and exits. 
Replace panics with one of CriticalIf, FatalIf and a return of error.
2018-04-19 17:24:43 -07:00
wd256 846f3e8f59 Disable "chunked" uploading by the GCS client for objects smaller than the chunk size. (#5835)
By disabling chunked uploading when the object size is less than the chunk size,
memory is not allocated unnecessarily.
2018-04-19 16:15:14 -07:00
Minio Trusted 75cc2ce9d8 Update yaml files to latest version RELEASE.2018-04-19T22-54-58Z 2018-04-19 22:58:31 +00:00
Andreas Auernhammer 5a16671f72 do not set response header X-Frame-Options for S3 requests (#5838)
This change removes the X-Frame-Options header - It should
not be set for S3 requests since it can break CORS.

Fixes #5813
2018-04-19 11:46:37 -07:00
Ashish Kumar Sinha 9ebb72aa99 Introduce new unauthenticated endpoint /metric (#5723) (#5829)
/metric exposes Promethus compatible data for scraping metrics

Fixes: #5723
2018-04-18 16:01:42 -07:00
wd256 9f31da5d57 Fix PutObject/CopyObject with metadata for GCS gateway (#5828)
Make sure to apply standard headers such as Content-Type,
Content-Disposition and Content-Language to the correct
GCS object attributes during object upload and copy operations.

Fixes: #5800
2018-04-18 21:40:48 +05:30
Harshavardhana 0e9c73cbcf update browser ui-assets (#5831) 2018-04-17 09:31:55 +05:30
Harshavardhana c332b4fb5e fix shadowing issue in logger (#5830) 2018-04-16 13:36:45 -07:00
Rushan ed5f2d899d Browser: Disable input validation styles on Firefox (#5822) 2018-04-16 10:47:03 -07:00
Harshavardhana adf9a9d300 Remove all unused variables and functions (#5823) 2018-04-15 19:26:04 +05:30
Harshavardhana 97a8d856b6 Vendorize new changes from minio-go repo (#5821)
- When policy is empty delete the bucket policy (#966) (04/13/18) <Harshavardhana>
- Add tests to check if ListObjects/V2 returns expected StorageClass (#963) (04/10/18) <Nitish Tiwari>
- Update get/setBucketPolicy methods to use files instead of pkg/policy (#959) (04/10/18) <Nitish Tiwari>
- avoid unnecessary stat call during single copy (#962) (04/06/18) <Andreas Auernhammer>
- avoid sending SSE-S3 header during GET requests. (#965) (04/05/18) <Andreas Auernhammer>
- Fix stream SSE uploads with S3 encrypt type (#960) (04/02/18) <Jesús Espino>
- Fix xml parsing error for RemoveObjects API (#949) (03/29/18) <poornas>
- Allow to upload empty files in stream based uploads (#958) (03/26/18) <Jesús Espino>
- Add missing doneCh in the example for removeobjects (#955) (03/26/18) <Alexandr Korsak>
- tests: Remove partial related tests (#957) (03/26/18) <Anis Elleuch>
- Add transport connection broken error to retry list (#956) (03/19/18) <poornas>
- [refactor]: simplify client encryption examples (#952) (03/19/18) <Andreas Auernhammer>
- Add tests for putObjectContentLanguage (#950) (03/15/18) <Harshavardhana>
- Add putObject/getObject() client side encryption examples (#948) (03/13/18) <Harshavardhana>
2018-04-15 11:39:48 +05:30
Nitish Tiwari 638f01f9e4 Generalize loadConfig method to avoid reading from disk (#5819)
As we move to multiple config backends like local disk and etcd,
config file should not be read from the disk, instead the quick
package should load and verify for duplicate entries.
2018-04-13 15:14:19 -07:00
Harshavardhana 57b8db2088 Cleanup help to bring concise self-help message (#5788) 2018-04-13 11:57:05 -07:00
Harshavardhana bcd54632f5 Fix broken 'minio update' command on Windows (#5816) 2018-04-13 11:51:03 -07:00
Nitish Tiwari 42f775af4f
Add cache config for configuration comparison (#5812) 2018-04-13 23:45:19 +05:30
Minio Trusted 75f35b926d Update yaml files to latest version RELEASE.2018-04-12T23-41-09Z 2018-04-12 23:45:16 +00:00
Andreas Auernhammer f60765ac93 add some security HTTP headers (#5814)
This change adds some security headers like Content-Security-Policy.
It does not set the HSTS header because Content-Security-Policy prevents
mixed HTTP and HTTPS content and the server does not use cookies.
However it is a header which could be added later on.

It also moves some header added by #5805 from a vendored file
to a generic handler.

Fixes ##5813
2018-04-12 15:57:41 -07:00
Harshavardhana 1f07545e2a
Improve init messages for distributed setup (#5786)
Fixes #5531
2018-04-12 15:43:38 -07:00
rwagner_inf e39de65367 Add security HTTP Headers (#5805)
Some HTTP security headers in Minio.
To avoid problems with XSS and Clickjacking attacks.

X-Frame-Options
X-Frame-Options response header improve the protection
of web applications against Clickjacking. It declares a
policy communicated from a host to the client browser
on whether the browser must not display the transmitted
content in frames of other web pages.

X-XSS-Protection
This header enables the Cross-site scripting (XSS) filter in your browser.
2018-04-12 14:30:06 -07:00
Harshavardhana bd8d6e3c4e Return bucketNameInvalid properly in B2 gateway (#5787)
Fixes #5758
2018-04-12 11:20:21 +05:30
Harshavardhana 4a874dfbc1
Ignore prefix renames when dest directory is not empty (#5798)
Also make sure to not modify the underlying errors from
layers, we should return the error as is and one object
layer should translate the errors.

Fixes #5797
2018-04-11 17:15:42 -07:00
Bala FA 650c6ee8fb event/target/webhook: treat all 2xx as success. (#5792)
Fixes #5769
2018-04-10 17:45:54 -07:00
kannappanr 57a3d9c16c
Modify fatalIf, startup and update message logging code (#5780)
Use a common logging framework to log fatalIf, startup, Info and Update
messages.
2018-04-10 09:37:14 -07:00
kannappanr cef992a395
Remove error package and cause functions (#5784) 2018-04-10 09:36:37 -07:00
Harshavardhana 217fb470a7 Add a check to check if disk is writable (#5662)
This check is a pre-emptive check to return
error early before we attempt to use the disk
for any other operations later.

refer #5645
2018-04-10 09:26:09 +05:30
Harshavardhana eb0deabd73 Fail when TLS is configured and HTTP endpoints are provided (#5771) 2018-04-10 07:40:23 +05:30
dingjs a706c21f70 Tranlsate disk-caching/README.md,nas.md,large-bucket/README.md to Chinese. (#5775) 2018-04-09 17:41:16 -07:00
Harshavardhana 41c31d10ca
Enable s3,gcs,manta,oss,b2 gateways as production ready (#5795) 2018-04-09 16:33:48 -07:00
Harshavardhana 1d31ad499f Make sure to re-load reference format after HealFormat (#5772)
This PR introduces ReloadFormat API call at objectlayer
to facilitate this. Previously we repurposed HealFormat
but we never ended up updating our reference format on
peers.

Fixes #5700
2018-04-09 22:55:41 +05:30
Krishna Srinivas ae8e863ff4 disk.String() represents just path and not URL when disk is a local disk (#5785) 2018-04-06 16:59:31 -07:00
Andreas Auernhammer da9f0e324e return AWS S3 compatible error for invalid but equal keys during key rotation (#5783)
This change let the server return the S3 error for a key rotation
if the source key is not valid but equal to the destination key.

This change also fixes the SSE-C error messages since AWS returns error messages
ending with a '.'.

Fixes #5625
2018-04-06 14:15:23 -07:00
Harshavardhana 73f7a98590 Enable go1.10.1 version (#5666) 2018-04-06 11:07:39 -07:00
Andreas Auernhammer 2f20d90465 set storage class in FS mode (#5779)
This change sets the storage class of the object-info if a storage
class was specified during PUT. The server now replies with the
storage class which was set during uploading the object in FS mode.

Fixes #5777
2018-04-06 20:50:02 +05:30
Dee Koder c30f75de28 Update screenshots. Use png instead of jpgs. (#5770) 2018-04-06 13:51:25 +05:30
Harshavardhana 861550446a
Allow upto 15 minutes of timeskew for RPC calls as well (#5774)
Default installations of cloned VMs on VMware like env
might experience serious problems with time skewing,
allow for a higher value instead of 3 seconds we are
moving to 15 minutes just like API level skew.

Access to internet and configuring ntp might not be possible,
in such situations providing atleast a 15 minute skew could
cater for majority of situations.
2018-04-05 23:55:31 -07:00
kannappanr f8a3fd0c2a
Create logger package and rename errorIf to LogIf (#5678)
Removing message from error logging
Replace errors.Trace with LogIf
2018-04-05 15:04:40 -07:00
kannappanr 91fd8ffeb7
Move http package inside cmd (#5776) 2018-04-05 14:19:27 -07:00
Nitish Tiwari 9eb94fe8c8 Fix StorageClass field in ListObject/ListObjectV2 response (#5766)
Fixes: #5754
2018-04-05 10:56:28 -07:00