Commit graph

17 commits

Author SHA1 Message Date
Harshavardhana 79bab6b561 XL: Operations on uploads.json should cater for disk being unavailable. (#2277) 2016-07-24 18:08:15 -07:00
Harshavardhana f248089523 api: Implement bucket notification. (#2271)
* Implement basic S3 notifications through queues

Supports multiple queues and three basic queue types:

1. NilQueue -- messages don't get sent anywhere
2. LogQueue -- messages get logged
3. AmqpQueue -- messages are sent to an AMQP queue

* api: Implement bucket notification.

Supports two different queue types

- AMQP
- ElasticSearch.

* Add support for redis
2016-07-23 22:51:12 -07:00
Harshavardhana a0635dcdd9 XL: Do not rely on getLoadBalancedQuorumDisks for NS consistency. (#2243)
The reason is any function relying on `getLoadBalancedQuorumDisks`
cannot possibly have an idempotent behavior.

The problem comes from given a set of N disks returning just a
shuffled N/2 disks.  In case of a scenario where we have N/2
number of failed disks, the returned value of `getLoadBalancedQuorumDisks`
is not equal to the same failed disks so essentially calls using such
disks might succeed or fail randomly at different intervals in time.

This proposal change is we move to `getLoadBalancedDisks()`
and use the shuffled N disks as a whole. Since most of the time we might
hit a good disk since we are not reducing our solution space. This
also provides consistent behavior for all the functions which rely
on shuffled disks.

Fixes #2242
2016-07-21 00:27:08 -07:00
Harshavardhana cdf1373f8e XL: Ignore and continue for cases when bucket does not exist. (#2205)
Fixes #2201
Fixes #2204
2016-07-13 13:44:33 -07:00
Harshavardhana dc3bafb194 XL: isQuorum rename as isDiskQuorum, word it properly. (#2196) 2016-07-13 00:29:48 -07:00
Anis Elleuch 5cd629adca XL/fs: DeleteVol should not return error cleaning multipart dir for errVolumeNotFound (#2188) 2016-07-12 10:07:32 -07:00
Harshavardhana 126865e8df XL/bucket: Remove bucket should cleanup incomplete uploads as well. (#2173)
This behavior is in accordance with S3.

Fixes #2170
2016-07-12 01:01:47 -07:00
Harshavardhana ca1b1921c4 XL: Implement ignore errors. (#2136)
Each metadata ops have a list of errors which can be
ignored, this is essentially needed when

  - disks are not found
  - disks are found but cannot be accessed (permission denied)
  - disks are there but fresh disks were added

This is needed since we don't have healing code in place where
it would have healed the fresh disks added.

Fixes #2072
2016-07-07 22:10:27 -07:00
Harshavardhana 8c0942bf0d XL: Remove usage of reduceErr and make it isQuorum verification. (#1909)
Fixes #1908
2016-06-18 00:27:51 +05:30
Bala FA 61598ed02f posix: return errFaultyDisk on I/O errors. (#1885)
When I/O error is occured more than allowed limit, posix returns
errFaultyDisk.

Fixes #1884
2016-06-08 22:02:10 -07:00
Harshavardhana 82fd907933 XL/PutObject: Handle all pending cases of DiskNotFound. 2016-06-03 11:40:44 -07:00
Harshavardhana fb95c1fad3 XL: Bring in some modularity into format verification and healing. (#1832) 2016-06-02 16:34:15 -07:00
Harshavardhana ae311aa53b XL: Cleanup, comments and all the updated functions. (#1830) 2016-06-01 16:43:31 -07:00
karthic rao 7278b90fe1 Adding defer to the lock (#1785) 2016-05-28 15:15:53 -07:00
Harshavardhana b2293c2bf4 XL: Rename, cleanup and add more comments. (#1769)
- xl-v1-bucket.go - removes a whole bunch of code.
- {xl-v1,fs-v1}-metadata.go - add a lot of comments and rename functions
   appropriately.
2016-05-28 15:15:53 -07:00
Harshavardhana 1e393c6c5b XL: Add new metadata for checksum. (#1743) 2016-05-28 15:15:53 -07:00
Harshavardhana 293d246f95 XL/FS: Rewrite in new format. 2016-05-28 15:15:53 -07:00