maxmustermann/minio
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3875 commits 18 branches 277 tags 146 MiB
Commit graph

568 commits

Author SHA1 Message Date
Krishna Srinivas b288eaddb3 xl: bit-rot algo was not set in get-object. (#3652) 
fixes #3650
 2017-01-30 14:25:28 -08:00
Krishnan Parthasarathi 2665aba555 Fail PutBucketPolicy if conditions are incompatible with actions. (#3659) 2017-01-30 09:20:16 -08:00
Krishnan Parthasarathi 9b6bcb30d9 Fix handling of StringNotEquals condition operator (#3660) 2017-01-30 09:18:10 -08:00
Krishnan Parthasarathi 864b8795aa heal: Should delete stale object parts before healing (#3649) 2017-01-30 00:45:56 -08:00
Bala FA cc1575f944 fix: do port availability check only on macOS. (#3654) 
On macOS, if a process already listens on 127.0.0.1:PORT, net.Listen() falls back
to IPv6 address ie minio will start listening on IPv6 address whereas another
(non-)minio process is listening on IPv4 of given port.
To avoid this error sutiation we check for port availability only for macOS.

Note: checkPortAvailability() tries to listen on given port and closes it.
It is possible to have a disconnected client in this tiny window of time.
 2017-01-30 00:44:36 -08:00
Krishnan Parthasarathi b408d0e87d Add aws:Referer condition key support. (#3641) 
This change implements bucket policy enhancements required to restrict access based on HTTP referer.
See https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html#example-bucket-policies-use-case-4 for more information.

Fixes #3540
 2017-01-29 19:45:11 -08:00
Anis Elleuch e1bc99e4fe xl: Fix GET of an empty multiparted object (#3646) 
GetObject returns unsatisfied range error when we try to download an object
uploaded using multipart mechanism.
 2017-01-27 10:51:02 -08:00
Alex Ellis 02194ee3c6 Fix issue with webhook events not closing Response body, leaving connections open with remote server (#3645) 2017-01-27 09:50:08 -08:00
Harshavardhana 85f2b74cfd jwt: Cache the bcrypt password hash. (#3526) 
Creds don't require secretKeyHash to be calculated
everytime, cache it instead and re-use.

This is an optimization for bcrypt.

Relevant results from the benchmark done locally, negative
value means improvement in this scenario.

```
benchmark                       old ns/op     new ns/op     delta
BenchmarkAuthenticateNode-4     160590992     80125647      -50.11%
BenchmarkAuthenticateWeb-4      160556692     80432144      -49.90%

benchmark                       old allocs     new allocs     delta
BenchmarkAuthenticateNode-4     87             75             -13.79%
BenchmarkAuthenticateWeb-4      87             75             -13.79%

benchmark                       old bytes     new bytes     delta
BenchmarkAuthenticateNode-4     15222         9785          -35.72%
BenchmarkAuthenticateWeb-4      15222         9785          -35.72%
```
 2017-01-26 16:51:51 -08:00
Krishna Srinivas 152cdf1c05 fs: Move traceError() to lower functions where possible. (#3633) 2017-01-26 15:40:10 -08:00
Krishna Srinivas 17dd1c19df cleanup: refactor common code between FS and XL listDirFactory. (#3639) 2017-01-26 15:39:22 -08:00
Anis Elleuch 8e49a3d047 Simplify running cmd.Main() for external tests (#3636) 
An external test that runs cmd.Main() has a difficulty to set cmd arguments
and MINIO_{ACCESS,SECRET}_KEY values, this commit changes a little the current
behavior in a way that helps external tests.
 2017-01-26 15:22:41 -08:00
Krishna Srinivas cccf77229d cleanup: Move code in *-multipart-common.go to *-multipart.go (#3638) 
The code in *-multipart-common.go is not common anymore.
 2017-01-26 12:51:12 -08:00
Harshavardhana dafdc74605 fs: if fs.json is empty ignore it while reading metadata. (#3634) 
This is needed so that we don't send wrong errors
on previously failed PutObject() which would have
left a stale `fs.json` entry.
 2017-01-26 10:19:07 -08:00
Krishna Srinivas 82373e3d50 fs: cleanup - do not cache size of metafiles (#3630) 
* Remove Size() method and size field from lock.LockedFile
* WriteTo method of fsMeta and uploadsV1 now takes concrete type *lock.LockedFile
 2017-01-25 12:29:06 -08:00
Harshavardhana d41dcb784b Move to blake2b-simd due to perf problems in golang.org/x/crypto 
Ref https://github.com/golang/go/issues/18563
 2017-01-24 18:07:34 -08:00
Krishnan Parthasarathi 0e693e0284 Add dry-run query param for HealFormat API (#3618) 2017-01-24 08:11:05 -08:00
Anis Elleuch fc880f9b23 admin: Enhance set credentials test (#3619) 
Add more test cases and ignore access and secret keys set from the env
 2017-01-24 08:08:36 -08:00
Krishna Srinivas cead24b0f7 miniobrowser: Bring Minio browser source into minio repo. (#3617) 2017-01-23 18:07:22 -08:00
Krishna Srinivas 8489f22fe2 signature-v2: Use request.RequestURI for signature calculation. (#3616) 
* signature-v2: Use request.RequestURI for signature calculation.

* Use splitStr instead of strings.Split
 2017-01-23 17:01:44 -08:00
Anis Elleuch fc6f804865 server-mux: Keep listening after Accept() err (#3613) 
Accept() can return errors like: `too many open files`, no need to totally quit listening in this case.
 2017-01-23 09:55:34 -08:00
Anis Elleuch d1d89116f1 admin: Add version to service Status API response (#3605) 
Add server's version field to service status API:

"version":{
	"version":"DEVELOPMENT.GOGET",
	"commitID":"DEVELOPMENT.GOGET"
}
 2017-01-23 08:56:06 -08:00
Anis Elleuch e1f64141a2 presign-v2: Compute signature on encoded URL path (#3612) 
Encode the path of the passed presigned url before calculating the signature. This fixes
presigning objects whose names contain characters that are found encoded in urls.
 2017-01-23 08:54:32 -08:00
Krishnan Parthasarathi 586058f079 Implement mgmt REST APIs to heal storage format. (#3604) 
* Implement heal format REST API handler
* Implement admin peer rpc handler to re-initialize storage
* Implement HealFormat API in pkg/madmin
* Update pkg/madmin API.md to incl. HealFormat
* Added unit tests for ReInitDisks rpc handler and HealFormatHandler
 2017-01-23 00:32:55 -08:00
Anis Elleuch 47358e3104 server-mux: Add tcp idle read timeout (#3607) 
Avoid many idle client connections i.e client didn't send any data until a given stipulated amount of time. 

Default chosen here is 30 seconds.
 2017-01-22 14:48:27 -08:00
Harshavardhana 3640c63289 server/mux: PeekProtocol() should return error and connection be closed. (#3608) 
For TLS peekProtocol do not assume the incoming request to be a TLS
connection perform a handshake() instead and validate.

Also add some security related defaults to `tls.Config`.
 2017-01-22 12:14:00 -08:00
Harshavardhana 51fa4f7fe3 Make PutObject a nop for an object which ends with "/" and size is '0' (#3603) 
This helps majority of S3 compatible applications while not returning
an error upon directory create request.

Fixes #2965
 2017-01-20 16:33:01 -08:00
Andrei Kopats c3f7d1026f fs: start even if there are not enough free space (#3606) 2017-01-20 09:30:20 -08:00
Harshavardhana 80f1387877 Initialize peers properly for localhost. (#3600) 
This introduced a regression.

Fixes #3594
 2017-01-19 11:32:13 -08:00
Andreas Kohn 0674fa43ff Handle the region for GetBucketLocation and PutBucket properly (#3596) 
This adjusts the code for these two handlers to match the logic in ListBucketHandler.

Fixes #3595
 2017-01-19 11:31:51 -08:00
Harshavardhana a17f1e875c server/mux: Close the connection even if buffer.Flush() returns error. (#3599) 
It is possible that buf.Flush() might return an error, leading to a
potential leak in active sockets.
 2017-01-19 11:19:57 -08:00
Jeffery Utter 9e1f1b50e0 Don't Check Available Inodes on NFS (#3598) 
In some cases (such as with VirutualBox, this value gets hardcoded
to 1000, which is less than the required minimum of 10000.

Fixes #3592
 2017-01-19 10:39:44 -08:00
Anis Elleuch 0715032598 heal: Add ListBucketsHeal object API (#3563) 
ListBucketsHeal will list which buckets that need to be healed:
  * ListBucketsHeal() (buckets []BucketInfo, err error)
 2017-01-19 09:34:18 -08:00
Harshavardhana dfc2ef3004 storage/rpc: Remove network error restriction. (#3591) 
This restriction has lots of side affects, since
we do not have a mechanism to clear states like
this it is better not to keep them.

Network errors are common and can occur with
simple cable removal etc. Since we already have
a retry mechanism this error count and stateful
nature can bring problems on a long running
cluster.
 2017-01-18 12:55:57 -08:00
Harshavardhana 62f8343879 Add constants for commonly used values. (#3588) 
This is a consolidation effort, avoiding usage
of naked strings in codebase. Whenever possible
use constants which can be repurposed elsewhere.

This also fixes `goconst ./...` reported issues.
 2017-01-18 12:24:34 -08:00
Anis Elleuch f803bb4b3d admin: Add service Set Credentials API (#3580) 2017-01-17 14:25:59 -08:00
Andy Brown 20a65981bd add delete and options methods to CORS whitelist (#3589) 2017-01-17 14:20:05 -08:00
Harshavardhana 09b450d610 Fix fs tests to avoid deleting /usr to certain systems. 2017-01-17 14:05:07 -08:00
Krishnan Parthasarathi c194b9f5f1 Implement mgmt REST APIs for heal subcommands (#3533) 
The heal APIs supported in this change are,
- listing of objects to be healed.
- healing a bucket.
- healing an object.
 2017-01-17 10:02:58 -08:00
Harshavardhana 98a6a2bcab obj: Return objectInfo for CompleteMultipartUpload(). (#3587) 
This patch avoids doing GetObjectInfo() in similar way
how we did for PutOject().
 2017-01-16 19:23:43 -08:00
Harshavardhana 1c699d8d3f fs: Re-implement object layer to remember the fd (#3509) 
This patch re-writes FS backend to support shared backend sharing locks for safe concurrent access across multiple servers.
 2017-01-16 17:05:00 -08:00
Harshavardhana b580ad24e2 server/http: Add missing keep alive for incoming tcp connections. (#3585) 
This was seen reproducing a bug with @gowithplanb. Windows Cloud
Berry clients do not close their respective connections.
 2017-01-16 03:38:06 -08:00
Harshavardhana f8a3b1e164 Fix a bug in previous patch. 2017-01-16 01:48:34 -08:00
Harshavardhana bf2b8879b7 config: Allow non-standard config dir to be configured with SSL. (#3583) 2017-01-15 16:53:01 -08:00
Harshavardhana caecd75a2a Deprecate and remove service stop API. (#3578) 
Fixes #3570
 2017-01-14 14:48:52 -08:00
Anis Elleuch 2959c104b3 peer rpc: Fix typo in cluster credentials update (#3579) 
Update credentials in cluster wasn't working due to a typo
 2017-01-14 14:06:23 -08:00
Anis Elleuch f64f8b03cb admin: Enhance locks list json response (#3573) 2017-01-13 14:25:34 -08:00
Harshavardhana 7b85756c64 notify/webhook: Handle webendpoints without port (#3568) 
Fixes and issue initializing webhook notification

```
FATA[0000] Initializing object layer failed              cause=Unable to initialize event \
    notification. dial tcp: missing port in address requestb.in source=[server-main.go:448:serverMain()]
```
 2017-01-12 20:08:00 -08:00
Alex d6a327fbc5 Add notifications by webhook. 
Add a new config entry moving to version 13.
```
		"webhook": {
			"1": {
				"enable": true,
				"address": "http://requestb.in/1i9al7m1"
			}
		}
```
 2017-01-12 10:19:59 -08:00
Anis Elleuch f24753812a nats: Add support of NATS.io Streaming server (#3494) 2017-01-11 16:41:05 -08:00