Commit graph

6061 commits

Author SHA1 Message Date
Harshavardhana
0cd0f6c255
Avoid error modification during IAM migration (#8156)
The underlying errors are important, for IAM
requirements and should wait appropriately at
the caller level, this allows for distributed
setups to run properly and not fail prematurely
during startup.

Also additionally fix the onlineDisk counting
2019-08-30 10:41:02 -07:00
poornas
4553db3872 mint: turn off progress for awscli sync test (#8158)
Bumping up to awscli 1.11.177 which supports --no-progress flag
2019-08-30 10:26:48 -07:00
Aditya Manthramurthy
847a3ea0a2 Add unit tests and refactor to improve coverage (#7617) 2019-08-29 13:53:27 -07:00
Aditya Manthramurthy
1f3d270de8 Fix delete policy routing (#8145) 2019-08-29 07:07:43 +05:30
Minio Trusted
daa57e8e5a Update yaml files to latest version RELEASE.2019-08-29T00-25-01Z 2019-08-29 00:30:50 +00:00
Aditya Manthramurthy
eb18c82976 Remove policy query param from being rejected for objects (#8144) 2019-08-28 16:58:40 -07:00
Krishna Srinivas
2ab0681c0c Do not ignore Lock()'s return value (#8142) 2019-08-28 16:12:57 -07:00
Harshavardhana
83d4c5763c
Decouple ServiceUpdate to ServerUpdate to be more native (#8138)
The change now is to ensure that we take custom URL as
well for updating the deployment, this is required for
hotfix deliveries for certain deployments - other than
the community release.

This commit changes the previous work d65a2c6725
with newer set of requirements.

Also deprecates PeerUptime()
2019-08-28 15:04:43 -07:00
Ashish Kumar Sinha
c6f86d35d3 Rectify Examples and README.MD (#8141) 2019-08-28 10:56:02 -07:00
Harshavardhana
d65a2c6725
Implement cluster-wide in-place updates (#8070)
This PR is a breaking change and also deprecates
`minio update` command, from this release onwards
all users are advised to just use `mc admin update`
2019-08-27 11:37:47 -07:00
Harshavardhana
70136fb55b
Look for network errors appropriately for RemoteStorageAPI (#8128)
net.Error is very unreliable in providing better error
handling, we need to ensure that we always have a fallback
option in case of network failures.

This fixes an important issue in our distributed server
setups when one of the servers is down, all deployments
out there are recommended to upgrade after this fix is
merged to ensure that availability is not lost.

Fixes #8127
Fixes #8016
Fixes #7964
2019-08-25 13:32:49 -07:00
Harshavardhana
d6dd98e597
Avoid data-race in getDisksInfo call (#8126) 2019-08-23 17:03:15 -07:00
Krishna Srinivas
c38ada1a26 write() to disk in 4MB blocks for better performance (#7888) 2019-08-23 15:36:46 -07:00
poornas
48bc3f1d53 Allow cached content to be encrypted (#8001)
If MINIO_CACHE_ENCRYPTION_MASTER_KEY is set,
automatically encrypt all cached content on disk.
2019-08-23 10:13:22 -07:00
Praveen raj Mani
e211f6f52e Parallelize the DiskInfo calls in xl.StorageInfo() (#8115) 2019-08-22 20:02:40 -07:00
Harshavardhana
f13f421e84
Allow CopyObject in pathStyle across federated instances (#8064)
Fixes #7976
2019-08-21 22:02:39 -10:00
Aditya Manthramurthy
cd03bfb3cf Fix ignoring claims in list buckets call (#8118) 2019-08-21 19:20:11 -10:00
Harshavardhana
add1b6cb6b
Remove go mod cache due to buggy travis caching (#8113)
There are sporadic failures due to go-mod build
caches on travis, deprecate it since we moved to
using GOPROXY.
2019-08-21 13:38:52 -10:00
Dee Koder
4f3c9d7677
Update doc with Object Tagging not supported (#8116) 2019-08-21 16:10:59 -07:00
poornas
2e19619e79 browser: Avoid logging BucketNotEmpty error (#8110) 2019-08-21 10:01:46 -10:00
Minio Trusted
de736f2864 Update yaml files to latest version RELEASE.2019-08-21T19-40-07Z 2019-08-21 19:46:01 +00:00
Harshavardhana
2fa98b1d6a Convert errAuthentication as AccessDenied appropriately (#8105)
Fixes #8062
2019-08-21 09:13:15 +05:30
kannappanr
99a4298938 Use a non-strict invalid bucket name check in Get and Delete object (#8073) 2019-08-20 17:40:52 -10:00
Harshavardhana
069badc7e9
Allow CopyObjectPart to work in federated setups (#8066)
Fixes #8065
2019-08-20 07:19:22 -10:00
Harshavardhana
c601cb2f1e
Add listBucketObjectsVersions implementation (#8093)
This API implementation simply behaves like listObjects()
but returns back single version for each object, this
implementation should be considered dummy it is only
meant for some applications which rely on this.
2019-08-19 11:02:54 -10:00
Harshavardhana
9ca7470ccc
Avoid using jsoniter, move to fastjson (#8063)
This is to avoid using unsafe.Pointer type
code dependency for MinIO, this causes
crashes on ARM64 platforms

Refer #8005 collection of runtime crashes due
to unsafe.Pointer usage incorrectly. We have
seen issues like this before when using
jsoniter library in the past.

This PR hopes to fix this using fastjson
2019-08-19 08:35:52 -10:00
Harshavardhana
b3ca304c01
Avoid excessive listing attempts in the daily sweep (#8081)
Add better dynamic timeouts for locks, also
add jitters before launching daily sweep to ensure
that not all the servers in distributed setup
are not trying to hold locks to begin the sweep
round.

Also, add enough delay for incoming requests based
on totalSetCount*totalDriveCount.

A possible fix for #8071
2019-08-19 08:22:32 -10:00
Bala FA
60f52f461f add network read performance collection support. (#8038)
ReST API on /minio/admin/v1/performance?perfType=net[?size=N] 
returns

```
{
  "PEER-1": [
             {
	       "addr": ADDR,
	       "readPerf": DURATION,
	       "error": ERROR,
	     },
	     ...
	   ],
  ...
  ...
  "PEER-N": [
             {
	       "addr": ADDR,
	       "readPerf": DURATION,
	       "error": ERROR,
	     },
	     ...
	   ]
}
```
2019-08-19 08:26:32 +05:30
Nitish Tiwari
e5fb6294a7 Remove healthcheck script for Docker image (#8095)
There are multiple possibilities for running MinIO within
a container e.g. configurable address, non-root user etc.
This makes it difficult to identify actual IP / Port to
use to check healthcheck status from within a container.

It is simpler to use external healthcheck mechanisms
like healthcheck command in docker-compose to check
for MinIO health status. This is similar to how checks
work in Kubernetes as well.

This PR removes the healthcheck script used inside
Docker container and ad documentation on how to
use docker-compose based healthcheck mechanism.
2019-08-17 12:44:04 -07:00
Harshavardhana
a15bb19d37
Allow audit logging to work while tracing (#8077)
It is observed that when `mc admin trace` is being
used due to ResponseWriter wrapper, we loose information
about statusCode,statusText for audit logging.

This PR fixes this behavior
2019-08-15 16:17:46 -07:00
poornas
7bf1caa0fe Fix broken link to go install docs (#8090) 2019-08-15 16:00:50 -07:00
Harshavardhana
6e7962bf35
Return if paths are empty in DeleteFileBulk (#8085)
This avoids a network call, also fixes an issue
when empty paths are passed the underlying call
fails with "405 Method Not Allowed".

This is reproducible when you are deleting a
non-existent object.

Fixes #8083
2019-08-15 13:15:49 -07:00
Aditya Manthramurthy
825e29f301 Check if user or group is disabled when evaluating policy (#8078) 2019-08-14 16:59:16 -07:00
Harshavardhana
cebeca3075 Fix multi-user doc to reflect the latest change 2019-08-14 14:12:23 -07:00
Minio Trusted
26640033b0 Update yaml files to latest version RELEASE.2019-08-14T20-37-41Z 2019-08-14 20:42:16 +00:00
Krishnan Parthasarathi
bbb56739bd Add User-Agent header with MinIO release details in http logs (#7843)
This would allow http log target server to distinguish between log
messages across different versions of MinIO deployments.
2019-08-14 11:43:43 -07:00
Nitish Tiwari
1cd801b2e9 Fix DeleteObjects() to remove renamed objects inside (#8072) 2019-08-14 11:15:25 -07:00
Aditya Manthramurthy
bf9b619d86 Set the policy mapping for a user or group (#8036)
Add API to set policy mapping for a user or group

Contains a breaking Admin APIs change.

- Also enforce all applicable policies
- Removes the previous /set-user-policy API

 Bump up peerRESTVersion

Add get user info API to show groups of a user
2019-08-13 13:41:06 -07:00
kannappanr
bc79b435a2 Doc: Fix PutObject API known limitations text (#7704) 2019-08-13 08:50:43 -07:00
maihde
0ed6daab59 fix: #8051 so that stale DNS entries are cleaned-up (#8053) 2019-08-13 08:49:26 -07:00
Praveen raj Mani
e96f19c867 Use default ports for the schemes when dialing (#7960)
Fixes #7946
2019-08-13 16:43:11 +05:30
Harshavardhana
aaef18b1a3 Fail for disks which overflow upon usage calculation (#8056)
Fixes #8035
2019-08-13 02:58:43 -07:00
Kaan Kabalak
a48a034e5a Make directory path in the header editable (#8018)
This change will allow users to navigate to their desired locations,
including buckets and directories that haven't been "created" yet

Fixes #7883

Add tests

Change tooltip wording

Migrate to Font Awesome 5 to use path icon

Fix sidebar not closing on mobile
2019-08-12 22:36:19 -07:00
Harshavardhana
bf8ec8ad73
Cleanup ui-errors and print proper error messages (#8068)
* Cleanup ui-errors and print proper error messages

Change HELP to HINT instead, handle more error
cases when starting up MinIO. One such is related
to #8048

* Apply suggestions from code review
2019-08-12 21:25:34 -07:00
Harshavardhana
8ce424bacd Enhance audit logging to capture responseTimes (#8067)
Audit logging requires to have

- timeToFirstByte
- timeToResponse

timing information
2019-08-12 20:32:34 -07:00
Anis Elleuch
cea3e3f7a6 browser: Add user-agent header filter to gorilla mux route (#8040)
When a peer client which higher version sends a request to a peer
server with lower version, the returned status code is 200 OK instead
of 405 code. The reason is that the peer client request reaches the
browser handler, which registers itself by '/minio' route but without
any other constraints. Adding filtering by user agent header to the
browser route so internal requests to old endpoints versions return
405 error code.
2019-08-12 17:05:30 -07:00
Harshavardhana
af36c92cab
With ListBuckets() access-list only buckets the user has access (#8037)
This is a behavior change from AWS S3, but it is done with
better judgment on our end to allow the listing of buckets only
which user has access to.

The advantage is this declutters the UI for users and only
lists bucket which they have access to.

Precursor for this feature to be applicable is a policy
must have the following actions

```
s3:ListAllMyBuckets
```
and
```
s3:ListBucket
```

enabled in the policy.
2019-08-12 10:27:38 -07:00
Jakob Ackermann
1b258da108 [web-router] update the white list for favicons (#8024) 2019-08-11 22:17:02 -07:00
Andreas Auernhammer
35427a017d fix type conversion in UpdateKey for Vault (#8058)
This commit fixes a type conversion in the `UpdateKey`
implementation of Vault.
2019-08-11 22:20:25 +05:30
Cary FitzHugh
a0715f3b59 Adding Notes on Swarm Network Aliases (#8054) 2019-08-10 21:25:01 +05:30