maxmustermann/minio
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3070 commits 18 branches 277 tags 146 MiB
Commit graph

76 commits

Author SHA1 Message Date
Harshavardhana
d2557bb538 XL: GetObject caching implemented for XL. (#2017) 
The object cache implementation is XL cache, which defaults
to 8GB worth of read cache. Currently GetObject() transparently
writes to this cache upon first client read and then subsequently
serves reads from the same cache.

Currently expiration is not implemented.
 2016-07-03 17:15:38 -07:00
Krishna Srinivas
3f2b4d9dc2 Show "https" in the "minio server export/" output if certificates are available. (#2065) 
fixes #2036
 2016-07-01 14:32:53 -07:00
Harshavardhana
796fe165c7 server: Minor command line doc change for XL. (#2022) 2016-06-29 02:05:57 -07:00
Harshavardhana
936a916e78 server: Add connection rate limiter for server. (#1977) 2016-06-26 03:18:07 -07:00
Harshavardhana
de21126f7e XL: Re-align the code again. 2016-06-02 01:54:06 -07:00
Bala FA
116b5607d7 server: fix to have readable timeout value (#1823) 2016-06-01 09:14:50 -07:00
karthic rao
1947ae198e Adding read nad write timeout for unresponsive client connectinos (#1809) 2016-05-31 11:53:21 -07:00
Andreas Linz
f5dfa895a5 Exit with error code if minio server fails to start (#1704) 
This commit replaces the call to `errorIf` with `fatalIf`, so that the
minio server exits with an non-zero exit status if something fails, e.g.
the port was already openend by another process.
 2016-05-20 02:05:52 -07:00
Harshavardhana
9472299308 logging: Log only for unhandled errors, remove all the debug logging. (#1652) 
This patch brings in the removal of debug logging altogether, instead
we bring in the functionality of being able to trace the errors properly
pointing back to the origination of the problem.

To enable tracing you need to enable "MINIO_TRACE" set to "1" or "true"
environment variable which would print back traces whenever there is an
error which is unhandled or at the handler layer.

By default this tracing is turned off and only user level logging is
provided.
 2016-05-16 14:31:28 -07:00
Harshavardhana
49141eb3e4 http: Remove minhttp package and use standard Golang http. (#1587) 
The functionality provided by minhttp will be implemented
cleanly through our own APIs. Since we are not going to
send SIGUSR2 and manage configuration in that manner, it
doesn't make sense to use minhttp.

Fixes #1586
 2016-05-10 18:03:00 -07:00
Harshavardhana
a20ccb1e83 server: Print proper endpoint, along with https if configured. (#1514) 
Fixes #1492
 2016-05-06 21:18:29 -07:00
Harshavardhana
7ae40eb1bb minhttp: Remove probe usage, move to golang error. (#1459) 
Fixes #1454
 2016-05-03 01:07:34 -07:00
Harshavardhana
4e34e03dd4 xl/fs: Split object layer into interface. (#1415) 2016-04-29 14:24:10 -07:00
Harshavardhana
5f80edf232 routers: Fix a crash while initializing network fs. (#1382) 
Crash happens when 'minio server filename' a file name is
provided instead of a directory on command line argument.

```
panic: runtime error: slice bounds out of range

goroutine 1 [running]:
panic(0x5eb460, 0xc82000e0b0)
	/usr/local/opt/go/libexec/src/runtime/panic.go:464 +0x3e6
main.splitNetPath(0x7fff5fbff9bd, 0x7, 0x0, 0x0, 0x0, 0x0)
	/Users/harsha/mygo/src/github.com/minio/minio/network-fs.go:49 +0xb7
main.newNetworkFS(0x7fff5fbff9bd, 0x7, 0x0, 0x0, 0x0, 0x0)
	/Users/harsha/mygo/src/github.com/minio/minio/network-fs.go:90 +0x20a
main.configureServerHandler(0xc82024e1c8, 0x5, 0xc8200640e0, 0x1, 0x1, 0x0, 0x0)
	/Users/harsha/mygo/src/github.com/minio/minio/routers.go:43 +0x6ce
main.configureServer(0xc82024e1c8, 0x5, 0xc8200640e0, 0x1, 0x1, 0x5)
	/Users/harsha/mygo/src/github.com/minio/minio/server-main.go:86 +0x67
```
 2016-04-25 18:10:40 -07:00
Harshavardhana
a98a7fb1ad Implement XL layer - preliminary work. 2016-04-25 12:47:31 -07:00
Harshavardhana
6bc17a3aea server: Attempt to increase max open files. (#1328) 2016-04-18 22:05:32 -07:00
Harshavardhana
30b0b4deba storage/server/client: Enable storage server, enable client storage. 2016-04-16 16:25:53 -07:00
Krishna Srinivas
3c48537f20 refactor: refactor code to separate fs into object-layer and fs layer. (#1305) 2016-04-16 16:25:53 -07:00
Harshavardhana
fbd02d530d web: Change /rpc to /webrpc 2016-04-08 01:46:46 -07:00
Harshavardhana
0479d4976b objectAPI: Fix object API interface, remove unnecessary structs. 
ObjectAPI changes.
```
ListObjects(bucket, prefix, marker, delimiter string, maxKeys int) (ListObjectsInfo, *probe.Error)
ListMultipartUploads(bucket, objectPrefix, keyMarker, uploadIDMarker, delimiter string, maxUploads int) (ListMultipartsInfo, *probe.Error)
ListObjectParts(bucket, object, uploadID string, partNumberMarker, maxParts int) (ListPartsInfo, *probe.Error)
CompleteMultipartUpload(bucket string, object string, uploadID string, parts []completePart) (ObjectInfo, *probe.Error)
```
 2016-04-03 15:25:01 -07:00
Harshavardhana
484ba91b08 config: Migrate to the new version. Remove backend details. 
Migrate to new config format v4.
```
{
	"version": "4",
	"credential": {
		"accessKey": "WLGDGYAQYIGI833EV05A",
		"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
	},
	"region": "us-east-1",
	"logger": {
		"console": {
			"enable": true,
			"level": "fatal"
		},
		"file": {
			"enable": false,
			"fileName": "",
			"level": "error"
		},
		"syslog": {
			"enable": false,
			"address": "",
			"level": "debug"
		}
	}
}
```

This patch also updates [minio cli spec](./minio.md)
 2016-04-02 17:29:31 -07:00
Harshavardhana
aaf97ea02c config/main: Re-write config files - add to new config v3 
- New config format.

```
{
	"version": "3",
	"address": ":9000",
    "backend": {
          "type": "fs",
          "disk": "/path"
    },
	"credential": {
		"accessKey": "WLGDGYAQYIGI833EV05A",
		"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
	},
	"region": "us-east-1",
	"logger": {
		"file": {
			"enable": false,
			"fileName": "",
			"level": "error"
		},
		"syslog": {
			"enable": false,
			"address": "",
			"level": "debug"
		},
		"console": {
			"enable": true,
			"level": "fatal"
		}
	}
}
```

New command lines in lieu of supporting XL.

Minio initialize filesystem backend.
~~~
$ minio init fs <path>
~~~

Minio initialize XL backend.
~~~
$ minio init xl <url1>...<url16>
~~~

For 'fs' backend it starts the server.
~~~
$ minio server
~~~

For 'xl' backend it waits for servers to join.
~~~
$ minio server
... [PROGRESS BAR] of servers connecting
~~~

Now on other servers execute 'join' and they connect.
~~~
....
minio join <url1> -- from <url2> && minio server
minio join <url1> -- from <url3> && minio server
...
...
minio join <url1> -- from <url16> && minio server
~~~
 2016-03-23 19:16:09 -07:00
Harshavardhana
72d364cbf2 [GH-1240] main: Print keys after init and full server initialization. 
Setting MINIO_ACCESS_KEY and MINIO_SECRET_KEY re-writes the values
in config properly, but the init message is not updated.

Fix it by delay printing keys until everything is properly
initialized.
 2016-03-17 20:01:52 -07:00
Krishna Srinivas
e5b411caf4 startup: do not fail for non-EADDRINUSE errors. Fixes #1234 2016-03-17 01:36:08 +05:30
Krishna Srinivas
010e775b17 startup: specify the network - tcp4/tcp6 for ListenTCP() 2016-03-10 17:17:47 +05:30
Harshavardhana
d740d3aaae Merge pull request #1208 from krishnasrinivas/port-check 
startup: do not start minio server if port is not free. Fixes #1207
 2016-03-09 09:25:05 -08:00
Krishna Srinivas
ea7ea427ca startup: do not start minio server if port is not free. Fixes #1207 2016-03-09 19:39:40 +05:30
Harshavardhana
d5057b3c51 accessPolicy: Implement Put, Get, Delete access policy. 
This patch implements Get,Put,Delete bucket policies

Supporting - http://docs.aws.amazon.com/AmazonS3/latest/dev/access-policy-language-overview.html

Currently supports following actions.

   "*":                             true,
   "s3:*":                          true,
   "s3:GetObject":                  true,
   "s3:ListBucket":                 true,
   "s3:PutObject":                  true,
   "s3:CreateBucket":               true,
   "s3:GetBucketLocation":          true,
   "s3:DeleteBucket":               true,
   "s3:DeleteObject":               true,
   "s3:AbortMultipartUpload":       true,
   "s3:ListBucketMultipartUploads": true,
   "s3:ListMultipartUploadParts":   true,

following conditions for "StringEquals" and "StringNotEquals"

   "s3:prefix", "s3:max-keys"
 2016-03-08 17:44:50 -08:00
Harshavardhana
164dfe2ec9 console: Fix console color printing on windows. 
Print colored accessKeyID and secretAccessKey are unreadable on windows
on command prompts and powershell.

Use the console package from minio client.
 2016-03-04 10:07:19 -08:00
Olivier Sallou
678585c513 use environment variables to set and override access and secret keys at server startup 2016-02-22 10:49:02 -08:00
Harshavardhana
91a092792a presigned: Fix a bug in presigned request verification. 
Additionally add Docker proxy configuration.
 2016-02-18 02:23:12 -08:00
Harshavardhana
dd9aaa855c web/rpc: Merge ports with API server. 
Fixes #1081 and #1130
 2016-02-17 20:28:15 -08:00
Harshavardhana
5a9333a67b signature: Rewrite signature handling and move it into a library. 2016-02-16 17:28:16 -08:00
Harshavardhana
62f6ffb6db xl: Moved to minio/minio - fixes #1112 2016-02-11 15:43:36 -08:00
Harshavardhana
f4c8120cf9 server: Remove max-buckets option and now max buckets is unlimited. 
minio server max-buckets option removed. min-free-disk option is
now a flag.
 2016-02-06 18:25:47 -08:00
Harshavardhana
8557cbc9b7 fs: Add granular locking. 2016-02-04 20:40:58 -08:00
Harshavardhana
7a3409c309 fs: Cleanup Golang errors to be called 'e' and probe to be called as 'err' 
- Replace the ACL checks back, remove them when bucket
  policy is implemented.
- Move FTW (File Tree Walk) into ioutils package.
 2016-02-04 13:43:52 -08:00
Harshavardhana
64b7da4686 web: GetObjectURL should check if file exists before generating URL. 
Fixes - https://github.com/minio/miniobrowser/issues/20
 2016-02-03 00:00:36 -08:00
Harshavardhana
454d71cafa expiry: Remove auto-expiry. 
Move the logic outside and use scripting, cronjob to delete files.

Fixes #1019
 2016-02-02 19:35:51 -08:00
Harshavardhana
df91661ec6 flags: Remove anonymous, ratelimit, json and web-address flags. 
- Web address now uses the port + 1 from the API address port directly.
- Remove ratelimiting, ratelimiting will be achieved if necessary through
  iptables.
- Remove json flag, not needed anymore.
- Remove anonymous flag, server will be no more anonymous for play.minio.io
  we will use demo credentials.
 2016-02-02 18:37:09 -08:00
Harshavardhana
d55f72f09a Add doc change. 2016-02-01 02:45:19 -08:00
Harshavardhana
a0c753b6eb server: Add new set of message. 2016-01-30 18:33:33 -08:00
frankw
0a5094c73a Added Alias argument in configuring/using mc client for minio server 
Both in README.md as well as sample commands when starting minio server
 2016-01-26 10:42:21 +01:00
Harshavardhana
3f1c4bb4b0 Bring in the list APIs implemented by Bala <bala@minio.io> 2016-01-24 16:39:48 -08:00
Harshavardhana
0a9496462a jwt: Add JWT support for minio server. 
Please read JWT.md before using this feature.
 2016-01-22 17:38:05 -08:00
Harshavardhana
1aec985c00 server: Fix shadowing bug reported by go vet on go1.6beta2 2016-01-18 18:45:24 -08:00
Harshavardhana
cb7b2762f9 serverConfig: Add a new region config entry. 
To change default region from 'us-east-1' to 'custom'.
Add a region value in your 'config.json'.

	"version": "2",
	"credentials": {
		"accessKeyId": "****************",
		"secretAccessKey": "***************",
	        "region": "my-region"
	},
 2016-01-17 01:39:11 -08:00
koolhead17
d4dbd09a9c Modified command for adding a host to minio. 2016-01-13 23:36:42 +05:30
Harshavardhana
523e75759d minio: Fix mc and minio download URLs. 
Fixes #991
 2015-12-13 13:57:56 -08:00
Harshavardhana
836f5204af minio: Add config-folder option. 
Fixes #997
 2015-12-07 12:34:09 -08:00