Commit graph

5070 commits

Author SHA1 Message Date
Andreas Auernhammer
cd152f404a replace os.Exit with panic for logger.CriticalIf (#6065)
This commit prevents complete server failures caused by
`logger.CriticalIf` calls. Instead of calling `os.Exit(1)`
the function now executes a panic with a special value
indicating that a critical error happend. At the top HTTP
handler layer panics are recovered and if its a critical
error the client gets an InternalServerError status code.

Further this allows unit tests to cover critical-error code
paths.
2018-06-25 13:51:49 -07:00
Praveen raj Mani
5fbdd70de9 Minio handle for list_objects_v2 with a blank ContinuationToken (#5940)
minio will now return an error for a blank continuation token in list_objects_v2,so as in s3.

Fixes #5931
2018-06-25 12:35:43 -07:00
Ashish Kumar Sinha
0bbdd02a57 Updating disk storage for FS/Erasure mode (#6081)
Updating the disk storage stats for FS/Erasure coded backend
2018-06-25 10:46:48 -07:00
Annanay Agarwal
78abe5234e Add functionality to make cache limit configurable (#5991) 2018-06-25 10:24:12 -07:00
Minio Trusted
f46ee54194 Update yaml files to latest version RELEASE.2018-06-22T23-48-46Z 2018-06-22 23:52:10 +00:00
Harshavardhana
6005dbf01f Fix broken doc links (#6068) 2018-06-22 09:14:41 +05:30
Nitish Tiwari
eb0e56ccf6 Update content-language field for Azure gateway (#6061)
Fixes https://github.com/minio/minio-go/issues/996
2018-06-21 09:46:45 -07:00
Dee Koder
c91abe6c4b Update with absolute path for images (#6060) 2018-06-21 08:29:12 +05:30
Aditya Manthramurthy
670b538dde Fix server crash when no system certificates are found (#6055) 2018-06-19 13:38:22 -07:00
Nitish Tiwari
186000328e Update StorageClass from Objectinfo (#6054)
See #6051
2018-06-19 11:22:08 -07:00
Harshavardhana
2575f4198a Auto-probe backend signature support (#6044) 2018-06-18 15:27:14 -07:00
Harshavardhana
05a64dee95 Do not leave stale entries on etcd on a failed update operation (#6048) 2018-06-18 12:05:17 -07:00
kannappanr
577d10674d
Cleanup stack trace in error logs (#6045)
Add compile time GOROOT path to the list of prefix
of file paths to be removed.

Add webhandler function names to the slice that
stores function names to terminate logging.
2018-06-18 12:04:46 -07:00
kannappanr
81ee79b042
Do not log browser login failures (#6049) 2018-06-18 11:49:49 -07:00
kannappanr
d94500ae26
Makefile: Create bin directory if it doesn't exist (#6050) 2018-06-18 11:45:28 -07:00
kannappanr
001d9a4ae7
Remove unwanted volume access denied log (#6042)
Fixes #6039
2018-06-18 11:43:41 -07:00
Harshavardhana
c3a5146422 Presign content-sha256 should look for query and header (#6046) 2018-06-15 14:21:17 -07:00
Nitish Tiwari
36c39d04da Update vendorized minio-go to support start-after param (#6043)
Fixes #6032
2018-06-14 15:08:02 -07:00
Harshavardhana
28d526bc68 Change CriticalIf to FatalIf for proper error message (#6040)
During startup until the object layer is initialized
logger is disabled to provide for a cleaner UI error
message. CriticalIf is disabled, use FatalIf instead.

Also never call os.Exit(1) on running servers where
you can return error to client in handlers.
2018-06-14 10:17:07 -07:00
Harshavardhana
05f96f3956 Avoid allocating larger buffer if the incoming body is small (#6035) 2018-06-13 11:55:12 -07:00
Harshavardhana
cb9ee1584a Fix TestHealStartNStatusHandler sporadic failure (#6015)
Fixes #5818
2018-06-12 16:36:31 -07:00
Andreas Auernhammer
9f4c120731 limit memory allocations during multiple object deletion (#6033)
This commit limits the amount of memory allocated by the
S3 Multi-Object-Delete-API. The server used to allocate as
many bytes as provided by the client using Content-Length.

S3 specifies that the S3 Multi-Object-Delete-API can delete
at most 1000 objects using a single request.
(See: https://docs.aws.amazon.com/AmazonS3/latest/API/multiobjectdeleteapi.html)
Since the maximum S3 object name is limited to 1024 bytes the
XML body sent by the client can only contain up to 1000 * 1024
bytes (excluding XML format overhead).

This commit limits the size of the parsed XML for the S3
Multi-Object-Delete-API to 2 MB. This fixes a DoS
vulnerability since (auth.) clients, MitM-adversaries
(without TLS) and un-auth. users accessing buckets allowing
multi-delete by policy can kill the server.

This behavior is similar to the AWS-S3 implementation.
2018-06-11 10:51:38 -07:00
Harshavardhana
12a916091e Convert federation doc as quickstart guide (#6030) 2018-06-09 14:00:11 +05:30
Minio Trusted
842092f8de Update yaml files to latest version RELEASE.2018-06-09T03-43-35Z 2018-06-09 03:47:19 +00:00
Harshavardhana
371349787f Remove region requirement for Healing (#6031) 2018-06-08 17:54:57 -07:00
Nitish Tiwari
3dc13323e5 Use random host from among multiple hosts to create requests
Also use hosts passed to Minio startup command to populate IP
addresses if MINIO_PUBLIC_IPS is not set.
2018-06-08 10:22:01 -07:00
Nitish Tiwari
6ce7265c8c Add support for CopyObject across regions and multiple Minio IPs
This PR adds CopyObject support for objects residing in buckets
in different Minio instances (where Minio instances are part of
a federated setup).

Also, added support for multiple Minio domain IPs. This is required
for distributed deployments, where one deployment may have multiple
nodes, each with a different public IP.
2018-06-08 10:22:01 -07:00
Nitish Tiwari
f30c95a301 Add etcd handling for web-handler methods 2018-06-08 10:22:01 -07:00
Harshavardhana
481390d51a Converge etcd functionality as part of quick.Config 2018-06-08 10:22:01 -07:00
Nitish
6df1e4a529 Add functionality to add old buckets to etcd on startup
Buckets already present on a Minio server before it joins a
bucket federated deployment will now be added to etcd during
startup. In case of a bucket name collision, admin is informed
via Minio server console message.

Added configuration migration for configuration stored in etcd
backend.

Also, environment variables are updated and ListBucket path style
request is no longer forwarded.
2018-06-08 10:22:01 -07:00
Harshavardhana
853ea371ce Bring etcd support for bucket DNS federation
- Supports centralized `config.json`
- Supports centralized `bucket` service records
  for client lookups
- implement a new proxy forwarder
2018-06-08 10:22:01 -07:00
Minio Trusted
7872c192ec Update yaml files to latest version RELEASE.2018-06-08T03-49-38Z 2018-06-08 03:52:34 +00:00
Kaan Kabalak
39919708d6 Fix regression issue where users couldn't click into folders (#6029)
This commit fixes the issue introduced in #6023

The clicks on folder names direct the users inside the folders; while
the clicks on file names don't download the files, and will open file
previews on a separate modal in the future. Additionally, when a file
has been selected using the checkbox, it can now be downloaded without
being inside a zip file.

Fixes #6026
2018-06-07 20:43:51 -07:00
Minio Trusted
7e12c3e8b9 Update yaml files to latest version RELEASE.2018-06-07T19-10-07Z 2018-06-07 19:13:18 +00:00
Kaan Kabalak
94ec6f374e NewUX: On filename click, select file instead of downloading (#6023)
This commit has been done according to @abperiasamy's feedback as we
are going to reserve the click on file icon to open the preview modal
in the future.

Also, when the user now selects a single file, the file itself is
downloaded instead of a .zip file containing the file.

Fixes #6019
2018-06-07 12:02:02 -07:00
Harshavardhana
617a6d8e47 Add request tracing support for other REST APIs (#6013) 2018-06-07 10:41:13 -07:00
Praveen raj Mani
c0cfe21c00 Ignore region in the case of admin API (#5919)
Admin API is not an S3 API and hence it is not required
to honor server region while validating admin API calls.

Fixes #2411
2018-06-07 10:37:31 -07:00
Bala FA
6a8bfcef1c remove separate file for posix utils. (#5948) 2018-06-07 12:31:40 +05:30
Harshavardhana
6138cae8e7 Persist MINIO_WORM as part of config.json (#6022) 2018-06-06 18:10:51 -07:00
Krishna Srinivas
113570b514 Refresh in-memory bucket policy cache every 5 minutes (#6007) 2018-06-06 12:52:56 -07:00
Bala FA
6a53dd1701 Implement HTTP POST based RPC (#5840)
Added support for new RPC support using HTTP POST.  RPC's 
arguments and reply are Gob encoded and sent as HTTP 
request/response body.

This patch also removes Go RPC based implementation.
2018-06-06 14:21:56 +05:30
Harshavardhana
9d41051e91 Relax runtime requirement do not fail, print warning (#6017) 2018-06-06 09:25:12 +05:30
Harshavardhana
3143454982 Deprecate and remove configurable disk usage check (#6016) 2018-06-05 18:53:44 -07:00
Harshavardhana
eafc15cd47 Fix presigned URL for access key with special characters (#6012)
Fixes #6011
2018-06-05 10:48:51 -07:00
Harshavardhana
6fb0604502 Allow usage check to be configurable (#6006) 2018-06-04 18:35:41 -07:00
Harshavardhana
df1b33013f Fix byte pool usage, use only one pool for all sets. (#5990) 2018-06-01 16:41:23 -07:00
Harshavardhana
537fd8c821 Bring nicer ui error if creds are missing in Gateway (#6003) 2018-06-01 15:45:11 -07:00
Krishna Srinivas
69b9d6fbee Add user-agent for azure blob requests (#6002) 2018-06-01 15:44:48 -07:00
Harshavardhana
c22b9d5d4d
Fix s3cmd issue with ACL handling (#5999)
With the implementation of dummy GET ACL handlers,
tools like s3cmd perform few operations which causes
the ACL call to be invoked. Make sure that in our
router configuration GET?acl comes before actual
GET call to facilitate this dummy call.
2018-05-31 19:43:50 -07:00
Harshavardhana
c247e603d2 terminate disk-usage routine in tests (#5998)
tests were written in the manner by editing internal
variables of fsObjects to mimic certain behavior from
APIs, but this is racy when an active go-routine is
reading from the same variable.

Make sure to terminate the go-routine if possible for
these tests.
2018-05-31 14:03:46 -07:00