Commit graph

987 commits

Author SHA1 Message Date
Krishnan Parthasarathi
c9619673fb Implement SetConfig admin API handler. (#3792) 2017-02-27 11:40:27 -08:00
Anis Elleuch
461b2bbd37 admin: Move SetCredentials from Service to Generic (#3805)
Setting credentials doesn't belong to service management API
anymore.
2017-02-25 11:06:08 -08:00
Krishnan Parthasarathi
2745bf2f1f Implement ServerConfig admin REST API (#3741)
Returns a valid config.json of the setup. In case of distributed
setup, it checks if quorum or more number of nodes have the same
config.json.
2017-02-20 12:58:50 -08:00
Anis Elleuch
7f86a21317 admin: Add ServerInfo API() (#3743) 2017-02-15 10:45:45 -08:00
Harshavardhana
22909c849e objcache: Return io.ReaderAt to avoid Seeking and Reading. (#3735) 2017-02-11 17:17:58 -08:00
Bala FA
8912b6bf3b trie: new package (#3729)
This implements a simple trie tree for minio server/tools.
This package borrows idea from
https://godoc.org/golang.org/x/text/internal/triegen.
2017-02-10 11:51:41 -08:00
Karthic Rao
a799351926 Fix macOS builds (#3728) 2017-02-10 00:37:22 -08:00
Harshavardhana
9df01035da Remove XL references in public docs to Erasure. (#3725)
Ref #3722
2017-02-09 23:26:44 -08:00
Krishnan Parthasarathi
e5773e11c6 Make minio server compile on OpenBSD, NetBSD, Solaris (#3719) 2017-02-08 22:27:35 -08:00
Krishnan Parthasarathi
ce9aa2f2b2 Add uptime to ServiceStatus (#3690) 2017-02-08 00:13:02 -08:00
Anis Elleuch
f612a7dd85 madmin: Fix a typo in Locks duration query name (#3673) 2017-02-01 11:46:49 -08:00
Krishnan Parthasarathi
0472e5c1e1 Change query param name to duration in list/clear locks API (#3664)
Following is a sample list lock API request schematic,

  /?lock&bucket=mybucket&prefix=myprefix&duration=holdDuration
  x-minio-operation: list

The response would contain the list of locks held on mybucket matching
myprefix for a duration longer than holdDuration.
2017-02-01 11:17:30 -08:00
Anis Elleuch
c753b5dfeb madmin: All APIs return ErrorResponse error type (#3632)
Avoid returning errors.New("Got HTTP Status") in all APIs
2017-01-25 16:01:29 -08:00
Krishna Srinivas
82373e3d50 fs: cleanup - do not cache size of metafiles (#3630)
* Remove Size() method and size field from lock.LockedFile
* WriteTo method of fsMeta and uploadsV1 now takes concrete type *lock.LockedFile
2017-01-25 12:29:06 -08:00
Anis Elleuch
76b8abcd8e madmin: Fix args order in listObjectsHeal() (#3629)
The order of marker and delimiter and in listObjectsHeal() internal function
are switched. That will give wrong result in case of a non recursive objects
heal list.
2017-01-25 09:58:49 -08:00
Krishnan Parthasarathi
0e693e0284 Add dry-run query param for HealFormat API (#3618) 2017-01-24 08:11:05 -08:00
Anis Elleuch
d1d89116f1 admin: Add version to service Status API response (#3605)
Add server's version field to service status API:

"version":{
	"version":"DEVELOPMENT.GOGET",
	"commitID":"DEVELOPMENT.GOGET"
}
2017-01-23 08:56:06 -08:00
Krishnan Parthasarathi
586058f079 Implement mgmt REST APIs to heal storage format. (#3604)
* Implement heal format REST API handler
* Implement admin peer rpc handler to re-initialize storage
* Implement HealFormat API in pkg/madmin
* Update pkg/madmin API.md to incl. HealFormat
* Added unit tests for ReInitDisks rpc handler and HealFormatHandler
2017-01-23 00:32:55 -08:00
Anis Elleuch
0715032598 heal: Add ListBucketsHeal object API (#3563)
ListBucketsHeal will list which buckets that need to be healed:
  * ListBucketsHeal() (buckets []BucketInfo, err error)
2017-01-19 09:34:18 -08:00
Anis Elleuch
f803bb4b3d admin: Add service Set Credentials API (#3580) 2017-01-17 14:25:59 -08:00
Krishnan Parthasarathi
c194b9f5f1 Implement mgmt REST APIs for heal subcommands (#3533)
The heal APIs supported in this change are,
- listing of objects to be healed.
- healing a bucket.
- healing an object.
2017-01-17 10:02:58 -08:00
Harshavardhana
1c699d8d3f fs: Re-implement object layer to remember the fd (#3509)
This patch re-writes FS backend to support shared backend sharing locks for safe concurrent access across multiple servers.
2017-01-16 17:05:00 -08:00
Harshavardhana
a054c73e22 Add slack replace gitter (#3584) 2017-01-16 15:26:26 -08:00
Harshavardhana
caecd75a2a Deprecate and remove service stop API. (#3578)
Fixes #3570
2017-01-14 14:48:52 -08:00
Anis Elleuch
f64f8b03cb admin: Enhance locks list json response (#3573) 2017-01-13 14:25:34 -08:00
Krishnan Parthasarathi
c8f57133a4 Implement list, clear locks REST API w/ pkg/madmin support (#3491)
* Filter lock info based on bucket, prefix and time since lock was held
* Implement list and clear locks REST API
* madmin: Add list and clear locks API
* locks: Clear locks matching bucket, prefix, relTime.
* Gather lock information across nodes for both list and clear locks admin REST API.
* docs: Add lock API to management APIs
2017-01-03 23:39:22 -08:00
Harshavardhana
f57f773189 admin: Add missing madmin examples and API docs. (#3483) 2016-12-20 18:49:48 -08:00
Harshavardhana
1d134c1a94 admin: Fix wrong vendorized location for s3signer and s3utils. 2016-12-20 15:18:37 -08:00
Anis Elleuch
329a910b86 Admin Lib: Implement Service API (#3426)
Three APIs were added to control a minio server

* NewAdminClient()
* ServiceStop()
* ServiceRestart()
* ServiceStatus()
2016-12-20 14:45:17 -08:00
Harshavardhana
b363709c11 caching: Optimize memory allocations. (#3405)
This change brings in changes at multiple places

 - Reuse buffers at almost all locations ranging
   from rpc, fs, xl, checksum etc.
 - Change caching behavior to disable itself
   under low memory conditions i.e < 8GB of RAM.
 - Only objects cached are of size 1/10th the size
   of the cache for example if 4GB is the cache size
   the maximum object size which will be cached
   is going to be 400MB. This change is an
   optimization to cache more objects rather
   than few larger objects.
 - If object cache is enabled default GC
   percent has been reduced to 20% in lieu
   with newly found behavior of GC. If the cache
   utilization reaches 75% of the maximum value
   GC percent is reduced to 10% to make GC
   more aggressive.
 - Do not use *bytes.Buffer* due to its growth
   requirements. For every allocation *bytes.Buffer*
   allocates an additional buffer for its internal
   purposes. This is undesirable for us, so
   implemented a new cappedWriter which is capped to a
   desired size, beyond this all writes rejected.

Possible fix for #3403.
2016-12-08 20:35:07 -08:00
Anis Elleuch
0cef971832 Fix max cache size calculation when system RAM is inferior to the default cache size (#3410) 2016-12-06 16:09:26 -08:00
Harshavardhana
1b85302161 Fix spelling and golint errors. (#3266)
Fixes #3263
2016-11-15 18:14:23 -08:00
Harshavardhana
2f373684f5 Fix the server startup messages and help text. (#3211) 2016-11-09 23:37:12 -08:00
Harshavardhana
f3c6c55719 posix: Fix windows performance issues. (#3132)
Do not attempt to fetch volume/drive information for
each i/o situation. In our case we do this in all calls
`posix.go` this in-turn created a terrible situation for
windows. This issue does not affect the i/o path on Unix
platforms since statvfs calls are in the range of micro
seconds on these platforms.

This verification is only needed during startup and we
let things fail at a later stage on windows.
2016-10-31 09:34:44 -07:00
Anis Elleuch
9417614a8e Recalculate free minimum disk space (#2788)
* Fix calculating free space disk by using blocks available for unprivileged user

* Use fixed minimal free disk space instead of percentage
2016-09-27 12:46:38 -07:00
Krishnan Parthasarathi
669783f875 Purge stale object cache entry (#2770) 2016-09-23 19:55:28 -07:00
Harshavardhana
7270ca4157 pkg/wildcard: Simplify the wildcard logic further. (#2555) 2016-08-27 00:27:17 -07:00
Harshavardhana
ec4260d260 api: BucketNotification should disallow duplicate notification. (#2539)
Added checks to look for duplicated notification configs.

Fixes #2472
2016-08-23 18:42:30 -07:00
Harshavardhana
4dec50ba51 build: Add platform specific fixes. 2016-08-16 14:40:41 -07:00
Harshavardhana
76d56c6ff2 typo: Fix typos across the codebase. (#2442) 2016-08-15 02:44:48 -07:00
Harshavardhana
7e46055a15 api/handlers: Implement streaming signature v4 support. (#2370)
* api/handlers: Implement streaming signature v4 support.

Fixes #2326

* tests: Add tests for quick/safe
2016-08-08 20:56:29 -07:00
Harshavardhana
b23605a2b5 pkg/objcache: Add more tests. (#2371) 2016-08-06 10:22:14 -07:00
Harshavardhana
064c51162d api: Add new ListenBucketNotificationHandler. (#2336)
This API is precursor before implementing `minio lambda` and `mc` continous replication.

This new api is an extention to BucketNofication APIs.

// Request
```
GET /bucket?notificationARN=arn:minio:lambda:us-east-1:10:minio HTTP/1.1
...
...
```

// Response
```

{"Records": ...}
...
...
...
{"Records": ...}
```
2016-08-04 22:01:58 -07:00
Harshavardhana
de5d5ff241 pkg/crypto: Deprecate cgo sha256 version. (#2354) 2016-08-04 03:19:36 -07:00
karthic rao
2e0742e309 bucket policy: Support for '?' wildcard. (#2353)
- Support for '?' wildcard for resource matching.

- Wildcard package is added with Match functions.

- Wildcard.Match supports '*' and wild.MatchExtended supports both '*'
  and '?' wildcards in the pattern string.

- Tests for the same for the wide range of cases.
2016-08-04 00:41:32 -07:00
Krishna Srinivas
69fd196471 Object-cache: enforce cache size to be less than RAM. (#2338) 2016-08-02 10:04:35 -07:00
Krishna Srinivas
043ddbd834 optimize memory allocation during erasure-read by using temporary buffer pool. (#2259)
* XL/erasure-read: optimize memory allocation during erasure-read by using temporary buffer pool.

With the change the buffer needed during GetObject by erasureReadFile is allocated only once.
2016-07-25 14:17:01 -07:00
Harshavardhana
c0c8a8430e XL/PutObject: Add single putObject and multipart caching. (#2115)
- Additionally adds test cases as well for object cache.
- Adds auto-expiry with expiration and cleanup time interval.

Fixes #2080
Fixes #2091
2016-07-08 20:34:27 -07:00
Krishna Srinivas
01cbacd803 object-cache: use golang bytes.Buffer and bytes.NewReader instead of custom implementation. (#2108) 2016-07-06 01:29:49 -07:00
Harshavardhana
8a028a9efb handler/PUT: Handle signature verification through a custom reader. (#2066)
Change brings in a new signVerifyReader which provides a io.Reader
compatible reader, additionally implements Verify() function.

Verify() function validates the signature present in the incoming
request. This approach is choosen to avoid complexities involved
in using io.Pipe().

Thanks to Krishna for his inputs on this.

Fixes #2058
Fixes #2054
Fixes #2087
2016-07-05 01:04:50 -07:00