Commit graph

32 commits

Author SHA1 Message Date
Harshavardhana
9dca46e156 signature: Use a layered approach for signature verification.
Signature calculation has now moved out from being a package to
top-level as a layered mechanism.

In case of payload calculation with body, go-routines are initiated
to simultaneously write and calculate shasum. Errors are sent
over the writer so that the lower layer removes the temporary files
properly.
2016-03-26 15:21:05 -07:00
Harshavardhana
52751d81cb cleanup: Rename ObjectMetadata as ObjectInfo.
Fixes #1215
2016-03-11 16:58:08 -08:00
Brendan Ashworth
294ea814bf pkg/fs: for locks, prefer defer and read-only ops
This commit prefers the use of 'defer' for fs.Unlock (and fs.RUnlock)
because it is more idiomatic Go and reduces repetition in the code,
lending to a cleaner code base.

It also switches a few uses of the lock to read-only locks, which should
improve performance of those functions dramatically in certain contexts.
2016-03-07 00:07:11 -08:00
Harshavardhana
aed62788d9 api: Implement multiple objects Delete api - fixes #956
This API takes input XML input in following form.

```
<?xml version="1.0" encoding="UTF-8"?>
<Delete>
    <Quiet>true</Quiet>
    <Object>
         <Key>Key</Key>
    </Object>
    <Object>
         <Key>Key</Key>
    </Object>
    ...
</Delete>
```

and responds the list of successful deletes, list of errors
for all the deleted objects.

```
<?xml version="1.0" encoding="UTF-8"?>
<DeleteResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Deleted>
    <Key>sample1.txt</Key>
  </Deleted>
  <Error>
    <Key>sample2.txt</Key>
    <Code>AccessDenied</Code>
    <Message>Access Denied</Message>
  </Error>
</DeleteResult>
```
2016-03-06 18:31:50 -08:00
Harshavardhana
f111997184 multipart: remove proper MD5, rather create MD5 based on parts to be s3 compatible.
This increases the performance phenominally.
2016-03-02 14:20:49 -08:00
Karthic Rao
6651f5b368 go vet shadow error patch 2016-03-02 09:55:00 +05:30
Harshavardhana
3ff8a1b719 api: Implement CopyObject s3 API, doing server side copy.
Fixes #1172
2016-02-27 19:51:59 -08:00
Anand Babu (AB) Periasamy
07da31f8b8 Merge pull request #1150 from harshavardhana/signature
signV4: Move pkg/signature to pkg/s3/signature4
2016-02-23 12:39:28 +05:30
Harshavardhana
653ceee9ee signV4: Move pkg/signature to pkg/s3/signature4
Cleanup and move this to relevant path.
2016-02-22 22:47:09 -08:00
Harshavardhana
800b19d8e5 cleanup: Remove definitions and move them to its relative places accordingly
- Move fs-definitions.go and break them into fs-datatypes.go, fs-bucket-acl.go
  and fs-utils.go
- Move api-definitions.go to api-response.go, where they should be.
- Move web-definitions to its related handlers.
2016-02-22 10:41:27 -08:00
Harshavardhana
5a9333a67b signature: Rewrite signature handling and move it into a library. 2016-02-16 17:28:16 -08:00
Harshavardhana
fbab7128d5 fs/multipart: Handle un-ordered creation of multiparts.
Fixes #1125
2016-02-14 00:39:15 -08:00
Harshavardhana
62f6ffb6db xl: Moved to minio/minio - fixes #1112 2016-02-11 15:43:36 -08:00
Harshavardhana
4e6e78598f multipart: Increase locked critical for CompleteMultipart. 2016-02-06 01:46:05 -08:00
Harshavardhana
8df201ef30 multipart: Multipart session map now is based on uploadID.
- Fixes initiating parallel uploads, and configs being quickly
  re-written by another incoming request.
- Parallel uploads work smoothly now and return expected behavior.
2016-02-05 23:32:30 -08:00
Harshavardhana
69bd001c8b multipart: Multipart resume simplify further. 2016-02-05 17:40:08 -08:00
Harshavardhana
6f80380497 fs: Use mimedb now. 2016-02-05 15:09:23 -08:00
Harshavardhana
a4c005ce30 multipart: Code cleanup
- More locking cleanup. Fix naming convention.
- Simplify concatenation and blocking calls.
2016-02-05 14:42:09 -08:00
Harshavardhana
8557cbc9b7 fs: Add granular locking. 2016-02-04 20:40:58 -08:00
Harshavardhana
7a3409c309 fs: Cleanup Golang errors to be called 'e' and probe to be called as 'err'
- Replace the ACL checks back, remove them when bucket
  policy is implemented.
- Move FTW (File Tree Walk) into ioutils package.
2016-02-04 13:43:52 -08:00
Harshavardhana
81fcbd2a54 statObject: Make sure to lowercase file extensions. 2016-02-02 17:54:59 -08:00
Harshavardhana
0aedb67de0 contentType: Reply back proper contentTypes based on the file extension.
Currently the server would set 'application/octet-stream' for all
objects, set this value based on the file extension transparently.

This is useful in case of minio browser to facilitate displaying
proper icons for the different mime data types.
2016-02-01 12:19:58 -08:00
Harshavardhana
db387912f2 jwt: Deprecate RSA usage, use HMAC instead.
HMAC is a much simpler implementation, providing the same
benefits as RSA, avoids additional steps and keeps the code
simpler.

This patch also additionally

- Implements PutObjectURL API.
- GetObjectURL, PutObjectURL take TargetHost as another
  argument for generating URL's for proper target destination.
- Adds experimental TLS support for JSON RPC calls.
2016-01-27 03:38:33 -08:00
Harshavardhana
2e311168ee multipart: NewMultipartUpload shouldn't return empty UploadID
Existing code
```
{
  if os.IsNotExist(e) {
       e = os.MkdirAll(objectDir, 0700)
       if e != nil {
            return "", probe.NewError(e)
       }
  }
  return "", probe.NewError(e)  ---> Error was here.
}
```
For a successful 'MkdirAll' it would still return an empty uploadID,
but the 'error' would be nil. This would succeed the request but
client would fail.

Fix is to re-arrange the logic. Thanks to Alexander Neumann @fd0, for
reporting this problem.
2016-01-26 15:00:34 -08:00
Harshavardhana
497f13d733 api: Various fixes.
- limit list buckets to limit only 100 buckets, all uppercase buckets
  are now lowercase and work transparently with all calls.
- Change disk.Stat to disk.GetInfo and return back disk.Info{} struct.
- Introduce new ioutils package which implements ReadDirN(path, n),
  ReadDirNamesN(path, n)
2016-01-25 16:08:27 -08:00
Harshavardhana
4fc161ddb1 bucketName: relax bucket names, now allow numbers as starting characters. 2015-12-07 13:58:55 -08:00
Harshavardhana
836f5204af minio: Add config-folder option.
Fixes #997
2015-12-07 12:34:09 -08:00
Harshavardhana
a328120e4d fs: Filter out $multiparts properly.
Relax md5 requirement during complete multipart upload - ref #977
2015-11-22 01:49:57 -08:00
Harshavardhana
35b9f965f1 fs: use new atomic package - use FileCreateWithPrefix() API 2015-11-17 16:32:20 -08:00
Harshavardhana
dddb1650de Add 5% cumulative reduction in total size of the disk
This is done due to filesystem holding additional metadata and inode space
which is unaccounted for during min-free-disk calculation.
2015-10-19 10:33:34 -07:00
Harshavardhana
5b2fa33bdb Implementing min-free-disk 2015-10-18 00:23:14 -07:00
Harshavardhana
762b798767 Migrate this project to minio micro services code 2015-10-16 11:26:08 -07:00