Commit graph

4918 commits

Author SHA1 Message Date
Andreas Auernhammer 9fb94e6aa8 ensure authenticated request bodies for Admin-API (#5984)
This commit adds a check to the server's admin-API such that it only
accepts Admin-API requests with authenticated bodies. Further this
commit updates the `madmin` package to always add the
`X-Amz-Content-Sha256` header.

This change improves the Admin-API security since the server does not
accept unauthenticated request bodies anymore.

After this commit `mc` must be updated to the new `madmin` api because
requests over TLS connections will fail.
2018-05-30 14:49:03 -07:00
Harshavardhana 5282639f3c Add prometheus endpoint to support total Used storageInfo (#5988)
Since we deprecated Total/Free we don't need to update
prometheus with those metrics. This PR also adds support
for caching implementation.
2018-05-30 11:30:14 -07:00
Arjun Mishra dd0db526d9 Remove quotes for XSS Protection Header (#5992) 2018-05-30 14:59:27 +05:30
Yaroslav Skopets a50cc7e937 Add Prometheus metrics for Minio gateway (#5987) 2018-05-30 10:13:46 +05:30
Praveen raj Mani 7ac0fccb6e updated pika API changes in rabbitMQ notification doc (#5980)
The pika python package used for listening on rabbitMQ events
has breaking change. 'type' param changed to 'exchange_type'
in newer releases.
2018-05-29 00:35:28 -07:00
Minio Trusted 3cdf601cf7 Update yaml files to latest version RELEASE.2018-05-25T19-49-13Z 2018-05-25 19:53:08 +00:00
Nitish Tiwari 5afd856355 Update Kubernetes TLS doc with info for distributed setups (#5971)
Also, add details on how to create wildcard self-signed certificates
using openssl
2018-05-23 20:41:25 -07:00
Harshavardhana 000e360196 Deprecate showing drive capacity and total free (#5976)
This addresses a situation that we shouldn't be
displaying Total/Free anymore, instead we should simply
show the total usage.
2018-05-23 17:30:25 -07:00
Harshavardhana e6ec645035 Implement support for calculating disk usage per tenant (#5969)
Fixes #5961
2018-05-23 15:41:29 +05:30
Acid Chicken (硫酸鶏) 483fe4bed5 Fix typo (#5960) 2018-05-22 08:09:30 +05:30
Harshavardhana 71c4ff9d10 Fix tabs/indents in help message (#5962) 2018-05-21 11:11:57 -07:00
Harshavardhana ac58283001 Support endpoints through env (#5952) 2018-05-18 17:51:03 -07:00
Andreas Auernhammer 9c8b7306f5 security: fix write-to-RAM DoS vulnerability (#5957)
This commit fixes a DoS vulnerability for certain APIs using
signature V4 by verifying the content-md5 and/or content-sha56 of
the request body in a streaming mode.

The issue was caused by reading the entire body of the request into
memory to verify the content-md5 or content-sha56 checksum if present.

The vulnerability could be exploited by either replaying a V4 request
(in the 15 min time frame) or sending a V4 presigned request with a
large body.
2018-05-18 11:27:25 -07:00
rawipfel 1cf381f1b0 handle Kubernetes read-only secrets (#5951) 2018-05-18 10:31:11 +05:30
kannappanr ffa6b45d00 Remove treewalk abort log message (#5953)
Log doesn't add any value and can be removed.

Fixes #5943
2018-05-17 17:26:16 -07:00
Bala FA 4eb788df79 rename checkPathValid() to getValidPath() (#5949) 2018-05-17 07:27:07 -07:00
Minio Trusted f9e8ac429e Update yaml files to latest version RELEASE.2018-05-16T23-35-33Z 2018-05-16 23:39:52 +00:00
Harshavardhana 64288d6eb5 fix: export policy.Resource for remote RPC policy updates (#5941)
This PR fixes a regression where the remote bucket policies
were not updated due to policy.Resource not having exported
fields, gob fails with unexported fields while marshalling.
2018-05-16 16:17:00 -07:00
Nitish Tiwari 41496e1406 Fix broken healthcheck link (#5935) 2018-05-16 14:43:25 -07:00
Nitish Tiwari e6ebcc4cb6 Remove redundant prometheus data points (#5934)
Removed field minio_http_requests_total as it was redundant with
minio_http_requests_duration_seconds_count

Also removed field minio_server_start_time_seconds as it was
redundant with process_start_time_seconds
2018-05-15 12:23:43 -07:00
Nitish Tiwari 9cab0f25e0 Add top level metrics document to summarize monitoring endpoints (#5923)
Minio server supports healthcheck and prometheus related
unauthenticated endpoints. This document summarizes this
information in a single place and add links for more detailed
documentation if needed.
2018-05-15 12:23:21 -07:00
Minio Trusted 5c21e89559 Update yaml files to latest version RELEASE.2018-05-11T00-29-24Z 2018-05-11 00:34:18 +00:00
Harshavardhana c872c30ea3 fix: introduce isLeafDir in healing to fix the crash (#5920)
This PR also supports healing directories.

Fixes #5917
2018-05-10 16:53:42 -07:00
Harshavardhana 5b74f918d4 Implement a dummy GetBucketACL for legacy reasons (#5906)
GetBucketACL call returns empty for all GET in ACL requests, 
the primary purpose of this PR is to provide legacy API support 
for legacy applications.

Fixes #5706
2018-05-10 09:32:26 +05:30
Minio Trusted 518f856900 Update yaml files to latest version RELEASE.2018-05-10T00-00-42Z 2018-05-10 00:03:30 +00:00
Krishna Srinivas cc8178cdc4 Log errors only once for event notification errors (#5905) 2018-05-09 15:59:45 -07:00
Harshavardhana 4886bfbc72 fix: Avoid more crashes due to concurrent map usage (#5912)
This PR fixes another situation where a crash occurs
thanks to @krishnasrinivas for reproducing this

Fixes #5897
2018-05-09 15:11:51 -07:00
Anis Elleuch c525424179 Return detailed msg for notification errors (#5913) 2018-05-09 15:11:24 -07:00
Anis Elleuch c1f1fa3cc8 More fixes for error messages (#5911) 2018-05-09 13:29:37 -07:00
Harshavardhana 98f81ced86 fix: Avoid concurrent map writes in go-routines (#5898)
Fixes #5897
2018-05-09 11:25:38 -07:00
Anis Elleuch 2c0020e9ee Fix typo in prometheus scrape url (#5909) 2018-05-09 11:20:03 -07:00
Praveen raj Mani 3971483e18 Remove trailing/leading spaces in Certificates (#5885)
Fixes:  #5632
2018-05-09 14:26:38 +05:30
Ashish Kumar Sinha deb685c5b5 Enhancements in Minio Prometheus exporter (#5848)
Standardized Minio collectors based on Prometheus 
recommendations.
2018-05-09 01:38:27 -07:00
Krishna Srinivas bb34bd91f1 Fix unnecessary log messages to avoid flooding the logs (#5900) 2018-05-09 01:38:27 -07:00
Krishna Srinivas 9de8fefa90 Fix azure list-objects not to list minio.sys.tmp/ (#5895) 2018-05-09 01:38:27 -07:00
Anis Elleuch 6d5f2a4391 Better support of empty directories (#5890)
Better support of HEAD and listing of zero sized objects with trailing
slash (a.k.a empty directory). For that, isLeafDir function is added
to indicate if the specified object is an empty directory or not. Each
backend (xl, fs) has the responsibility to store that information.
Currently, in both of XL & FS, an empty directory is represented by
an empty directory in the backend.

isLeafDir() checks if the given path is an empty directory or not,
since dir listing is costly if the latter contains too many objects,
readDirN() is added in this PR to list only N number of entries.
In isLeadDir(), we will only list one entry to check if a directory
is empty or not.
2018-05-09 01:38:21 -07:00
Anis Elleuch 32700fca52 Enhance fatal errors printing of common issues seen by users (#5878) 2018-05-08 19:04:36 -07:00
Harshavardhana 54cd29b51c Add new S3 APIs which are not implemented (#5899)
Fixes #5896
2018-05-08 21:45:10 +05:30
Harshavardhana c98d8cb1c7 fs: fix a regression allow reading of existing files (#5889) 2018-05-07 17:00:44 -07:00
Minio Trusted 2ecc976950 Update yaml files to latest version RELEASE.2018-05-04T23-13-12Z 2018-05-04 23:16:52 +00:00
Andreas Auernhammer c5a00e513c fix DoS vulnerability in request authentication (#5887)
This commit fixes a DoS vulnerability in the
request authentication. The root cause is an 'unlimited'
read-into-RAM from the request body.

Since this read happens before the request authentication
is verified the vulnerability can be exploit without any
access privileges.

This commit limits the size of the request body to 3 MB.
This is about the same size as AWS. The limit seems to be
between 1.6 and 3.2 MB - depending on the AWS machine which
is handling the request.
2018-05-04 11:16:14 -07:00
Anis Elleuch 9439dfef64 Use defer style to stop tickers to avoid current/possible misuse (#5883)
This commit ensures that all tickers are stopped using defer ticker.Stop()
style. This will also fix one bug seen when a client starts to listen to
event notifications and that case will result a leak in tickers.
2018-05-04 10:43:20 -07:00
Krishna Srinivas 0f746a14a3 Do not use crypto.SHA3_256 as placeholder for HighwayHash256 (#5847) 2018-05-04 10:42:22 -07:00
Harshavardhana d69ba7d085 fs: fix logging by not logging common scenarios (#5882)
Fixes #5881
2018-05-03 08:29:38 +05:30
Krishna Srinivas d6df9b16ac Return NoSuchKey for anonReqs with s3:ListBucket policy (#5876) 2018-05-02 12:13:27 +05:30
Krishna Srinivas 0ad0969d61 Remove s3:ListObjects policy action to be in sync with AWS-S3 (#5875) 2018-05-02 12:13:17 +05:30
wd256 7f207156ec Copy an object without user-defined metadata for Azure gateway (#5872) 2018-05-02 10:20:00 +05:30
Andreas Auernhammer ca8520fb31 vendor x/sys/cpu and update x/crypto/blake2b (#5870)
This change updates the vendor'd x/crypto/blake2b package and
adds x/sys/cpu.

This change fixes an issue for Go1.11 affecting x/crypto/blake2b
since G1.11 removes the runtime-internal functions `suports_avx()` and
`supports_avx2()`.
2018-05-01 11:16:56 +05:30
Harshavardhana 5f9041571f Heal only when atleast one of the disk is unformatted (#5866)
Current healing has an issue when disks are healed
even when they are offline without knowing if disk
is unformatted. This can lead to issues of pre-maturely
removing the disk from the set just because it was
temporarily offline.

There is an increasing number of `mc admin heal` usage
on a cron or regular basis. It is possible that if healing
code saw disk is offline it might prematurely take it down,
this causes availability issues.

Fixes #5826
2018-05-01 09:07:39 +05:30
cbenien e43cd0d5c8 Increased MaxIdleConnsPerHost to prevent excessive re-connections and TIME_WAIT when more than 100 clients are using minio (#5860) 2018-04-30 11:59:41 -07:00