maxmustermann/minio
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
2434 commits 18 branches 277 tags 146 MiB
Commit graph

877 commits

Author SHA1 Message Date
Harshavardhana baf66988e9 Simplify bucket delete - remove only bucket directory, no need to recursively traverse 2015-10-30 16:03:18 -07:00
Krishna Srinivas 0010f0ee10 Remove empty directories while removing an oobject 2015-10-29 23:30:16 -07:00
Harshavardhana 53adfb38f4 Leverage sort Interface to provide sortUnique function 2015-10-23 15:55:41 -07:00
Harshavardhana dbaa4d8643 Move ListObjects into its own file 2015-10-22 15:39:04 -07:00
Harshavardhana 1f66f4869b Fix all remaining windows path issues. 2015-10-22 00:05:10 -07:00
Harshavardhana afa27b9847 On windows translate Prefix, Marker and Delimiter for paths 2015-10-21 22:00:03 -07:00
Harshavardhana b74852116a Fix portability issues for arm on raspberry pi 2015-10-20 11:22:00 -07:00
Harshavardhana dddb1650de Add 5% cumulative reduction in total size of the disk 
This is done due to filesystem holding additional metadata and inode space
which is unaccounted for during min-free-disk calculation.
 2015-10-19 10:33:34 -07:00
Harshavardhana 179d2d7dac Add initial cut of auto expiry of objects 2015-10-19 01:34:31 -07:00
Harshavardhana c065be656c Implement min-free-disk as a subcommand, deprecate flag 2015-10-19 00:59:20 -07:00
Harshavardhana 5b2fa33bdb Implementing min-free-disk 2015-10-18 00:23:14 -07:00
Harshavardhana a8a935f5fd Improve disk code to return back disk StatFS{} structure 
```
StatFS {
Total int64
Free int64
FSType string
}
```

Provides more information in a cross platform way.
 2015-10-17 20:19:26 -07:00
Harshavardhana aee0845b2e Add disk package 
Implements

   - Stat returns total and free disk space supported across platforms
   - Type returns type of the filesystem underneath
 2015-10-17 16:48:24 -07:00
Harshavardhana 1256ca86d0 Add fs separator 2015-10-17 12:05:12 -07:00
Harshavardhana 2ec679a089 If directory already removed, return nil and move on 2015-10-17 00:13:46 -07:00
Harshavardhana d534fc5a4f Implement delete bucket properly with proper error handlings 2015-10-17 00:01:12 -07:00
Harshavardhana c9af01d807 Enhance listing further, this time handle cases related to common prefixes 2015-10-16 23:11:41 -07:00
Harshavardhana 0eb7f078f9 Implement Bucket ACL support 2015-10-16 19:47:30 -07:00
Harshavardhana 762b798767 Migrate this project to minio micro services code 2015-10-16 11:26:08 -07:00
Harshavardhana 3318cba132 Add windows support for minhttp library 2015-10-11 01:08:16 -07:00
Harshavardhana 7fea9cb550 Add rpc signature handler 2015-10-08 22:28:11 -07:00
Harshavardhana 11048708bb Implement GetBucketACL - fixes #893 2015-10-08 11:12:44 -07:00
Anand Babu (AB) Periasamy bf901d3b9a Merge pull request #895 from abperiasamy/tasker 
new task model minio server
 2015-10-08 02:23:29 -07:00
Anand Babu (AB) Periasamy b52697e6ad new task model minio server 2015-10-08 02:20:24 -07:00
Harshavardhana a060b158c8 Add quick.CheckVersion() to verify config version quickly before unmarshalling the full struct 
This is needed during migration where we would need to verify the underlying version number
in a quick way.
 2015-10-07 17:44:33 -07:00
Harshavardhana ab5ea997ab Make erasure Encode and Decode atomic to avoid races 2015-10-06 23:05:01 -07:00
Harshavardhana 1b42398e8b Canonicalize all the incoming input values, now PresignedPostPolicy works with minio-go 2015-10-06 10:21:28 -07:00
Harshavardhana d5ce2f6944 Make erasure matrix type not optional choose automatically 
Remove option of providing Technique and handling errors based on that
choose a matrix type automatically based on number of data blocks.

INTEL recommends on using cauchy for consistent invertible matrices,
while vandermonde is faster we should default to cauchy for large
data blocks.
 2015-10-05 22:38:02 -07:00
Harshavardhana 4ed50a8004 Fix encoding bug in donut during encoding phase 
Stream reading needs to check for length parameter being non zero,
after Reading() a predefined set of buffer length an EOF might be returned
with length == 0.

Erasure taking this zeroed data in might wrongly encode it as part of existing
data blocks which leads to errors while decoding even when the other contents
are intact.
 2015-10-05 22:12:53 -07:00
Harshavardhana f0a8dbecae Add --json output formatter for server 2015-10-05 00:20:49 -07:00
Harshavardhana c4faf47e64 First time mode for controller 
- Upon first time invocation ``minio controller`` would create access keys and secret id
- Upon request passing 'keys' arg ``minio controller`` would provide the keys
- Add colorized notification
 2015-10-04 16:42:16 -07:00
Harshavardhana cfdb29cac0 Simplify signature handling 
This change brings a new SignatureHandler where Presigned.
Requests without Payload are handled very early before even
going through the call.

This change simplifies Donut codebase to not have signature related
logic for all API's.

Simplification is still needed for Payload based signature eg. PUT/POST calls
, which are still part of the donut codebase, which will be done subsequently
after donut re-write.
 2015-10-04 13:15:33 -07:00
Harshavardhana c8de5bad2f Implement presigned policy 2015-10-04 00:01:34 -07:00
Anis ELLEUCH b5ea05d839 A better way to print prettified json syntax error msg 2015-10-03 12:25:44 -07:00
Harshavardhana 3b070dee16 Fix an important metadata getObject bug in donut 2015-10-01 10:18:03 -07:00
Harshavardhana 81cc017f91 Implement presigned signature v4 support 2015-10-01 10:17:47 -07:00
Harshavardhana 50750efb52 Reduce memory usage for memory multipart write by doing io.Pipe() streaming copy 2015-09-30 20:53:30 -07:00
Harshavardhana 8c7c5df770 Prober stringer should avoid frivolous newlines 2015-09-29 10:13:11 -07:00
Harshavardhana cd52d7a11c Fix Linux/Mac OS X erasure decoding failure with new Golang version 1.5.1 
Fixes #871
 2015-09-24 21:42:59 -07:00
Harshavardhana 3785489153 Disable multipart for donut backend from being used 
Will enable it later after cleanup
 2015-09-24 18:54:30 -07:00
Harshavardhana b938e40fb5 Move atomic package to the top and simplify its tests 2015-09-20 13:51:38 -07:00
Harshavardhana d54488f144 Move all server and controller packages into top-level 2015-09-19 01:07:42 -07:00
Anand Babu (AB) Periasamy 89a86948b5 new version format and some cleanup 2015-09-18 23:33:28 -07:00
Anand Babu (AB) Periasamy d1f1b7ac31 new version format and some cleanup 2015-09-18 23:27:04 -07:00
Harshavardhana 778f8cd222 Add new rpc tests for Server.Add and Server.List, improve Version.Get RPC to provide more details 2015-09-18 17:44:46 -07:00
Harshavardhana 2a15dd5eab Enhance signature handler - throw back valid error messages 2015-09-18 15:14:55 -07:00
Harshavardhana ac93bbb41d Merge pull request #846 from harshavardhana/new-changes 
With new auth config changes, restructure the API code to use the new style
 2015-09-18 03:45:03 -07:00
Harshavardhana 6a5e5c1826 With new auth config changes, restructure the API code to use the new style 2015-09-18 03:41:05 -07:00
Harshavardhana b4ce1e8c1d Add missing reply.Name and add possible failure tests 2015-09-18 03:15:19 -07:00
Harshavardhana f8bb85aeb7 Enhance auth JSONRPC, now provides persistent output 
Implements

   - Auth.Generate("user")
   - Auth.Fetch("user")
   - Auth.Reset("user")

This patch also adds testing for each of these cases
 2015-09-18 03:02:39 -07:00
Harshavardhana 4bcd86408b Fix minio header in accordance with rfc2616.txt 2015-09-17 23:46:10 -07:00
Harshavardhana 7093a05ab1 Version is a package now, will be re-used across codebase. 2015-09-17 20:17:33 -07:00
Harshavardhana 1887114444 Fix all the golint complaints about newly added changes 
Do not use func(this *server), such generic names should not be used
for writing struct methods.
 2015-09-17 18:53:42 -07:00
Harshavardhana 03ef6533c8 Add more documentation for probe 2015-09-17 18:10:42 -07:00
Krishna Srinivas c49407ced4 rename rpc/server.go -> rpc/rpc.go. rpc/server.go will accomodate ServerService 2015-09-17 15:46:51 -07:00
Harshavardhana 3f4b98ca4c Consolidate controller, move rpc package into controller - remove dangling code in pkg/server 2015-09-15 19:38:36 -07:00
Harshavardhana 8d5f6e0b96 Merge pull request #826 from krishnasrinivas/middleware-cleanup 
Remove unneeded functions in middleware init
 2015-09-15 18:11:45 -07:00
Krishna Srinivas b1b387b157 Remove unneeded functions in middleware init 2015-09-15 18:09:09 -07:00
Harshavardhana 45d8898019 Merge branch 'parallel-read2' of https://github.com/krishnasrinivas/minio into krishnasrinivas-parallel-read2 
Make few more changes and rebased with current master
 2015-09-15 17:33:33 -07:00
Anand Babu (AB) Periasamy 6930e4d668 logrus logger 2015-09-09 20:28:09 -07:00
Harshavardhana 1e2c010174 Avoid shadowing variables and enable checks to avoid them during build 2015-09-09 15:14:55 -07:00
Harshavardhana b649eff3fb Disable logging for now 2015-09-05 21:28:35 -07:00
Harshavardhana d0f945f8e7 Simplify erasure package for OSX 2015-09-05 20:19:43 -07:00
Harshavardhana afff3f8885 Revert "Enable controller to have CORS" 
This reverts commit f39ac24e99.
 2015-08-31 17:15:49 -07:00
Krishna Srinivas 1e82ee1192 Add CORS support to minio s3 server 2015-08-31 16:59:52 -07:00
Harshavardhana f39ac24e99 Enable controller to have CORS 2015-08-31 01:47:05 -07:00
Harshavardhana dcf0c71ca3 Merge pull request #810 from harshavardhana/restructure 
Restructure server code, controller now runs in silo
 2015-08-27 17:10:54 -07:00
Harshavardhana 025f95b1d6 Restructure server code, controller now runs in silo 2015-08-27 17:07:32 -07:00
Anand Babu (AB) Periasamy c11aa1c892 return *probe.Error for Untrace() as well. 2015-08-24 03:35:24 -07:00
Harshavardhana 988d39a5b6 Migrate to golang1.5 release with GO15VENDOREXPERIMENT=1 enabled 2015-08-22 18:35:37 -07:00
Harshavardhana 0e416ea699 Tests were running 4 times due to multiple times the TestingT{} was being called 
Calling multiple times TestingT{} will hook up runner for Suites for that many times
which would lead to repeated running tests.

Fix it by only initializing it once for all the Suites
 2015-08-20 22:32:50 -07:00
Harshavardhana 068d1d1ba9 Add a new quick.Load() function to load directly any config file provided a quick compatible struct{} is also provided 2015-08-20 20:33:49 -07:00
Anand Babu (AB) Periasamy b49b8cdbe8 return call stack in reverse 2015-08-19 22:40:27 -07:00
Anand Babu (AB) Periasamy 826202716f skip Env in json printinf if empty 2015-08-19 01:21:28 -07:00
Anand Babu (AB) Periasamy 76c40e075a make tracePoint public as well 2015-08-19 01:02:39 -07:00
Harshavardhana ac928b5092 Fix typo in probe 2015-08-18 23:42:41 -07:00
Anand Babu (AB) Periasamy cdf93e534c simplify probe APIs 2015-08-18 19:30:17 -07:00
Harshavardhana e9c5a51bc6 Migrate pkg/quick from mc 2015-08-13 16:29:55 -07:00
Harshavardhana 61175ef091 Migrate to govendor to avoid limitations of godep 
- over the course of a project history every maintainer needs to update
  its dependency packages, the problem essentially with godep is manipulating
  GOPATH - this manipulation leads to static objects created at different locations
  which end up conflicting with the overall functionality of golang.

  This also leads to broken builds. There is no easier way out of this other than
  asking developers to do 'godep restore' all the time. Which perhaps as a practice
  doesn't sound like a clean solution. On the other hand 'godep restore' has its own
  set of problems.

- govendor is a right tool but a stop gap tool until we wait for golangs official
  1.5 version which fixes this vendoring issue once and for all.

- govendor provides consistency in terms of how import paths should be handled unlike
  manipulation GOPATH.

  This has advantages
    - no more compiled objects being referenced in GOPATH and build time GOPATH
      manging which leads to conflicts.
    - proper import paths referencing the exact package a project is dependent on.

 govendor is simple and provides the minimal necessary tooling to achieve this.

 For now this is the right solution.
 2015-08-12 19:24:57 -07:00
Harshavardhana 45b59b8456 Probe revamped to provide for a new WrappedError struct to wrap probes as error interface 
This convenience was necessary to be used for golang library functions like io.Copy and io.Pipe
where we shouldn't be writing proxies and alternatives returning *probe.Error

This change also brings more changes across code base for clear separation regarding where an error
interface should be passed encapsulating *probe.Error and where it should be used as is.
 2015-08-08 00:16:38 -07:00
Harshavardhana f8141493bd use command not found helper 2015-08-03 18:06:19 -07:00
Harshavardhana d09fd8b0a1 Migrate from iodine to probe 2015-08-03 16:33:44 -07:00
Harshavardhana 884e9771b2 More changes to probe to avoid nil dereferences 2015-08-03 01:47:37 -07:00
Harshavardhana 65e4aede82 Minor changes to probe 2015-08-02 20:33:49 -07:00
Anand Babu (AB) Periasamy 697009c0a1 trace on New and add read locks 2015-08-02 11:58:28 -07:00
Harshavardhana d9493909d8 Merge pull request #788 from krishnasrinivas/open-and-openfile 
rename disk.OpenFile to Open which will do os.Open (which will be rea…
 2015-08-02 11:05:04 -07:00
Krishna Srinivas ee4432bc40 rename disk.OpenFile to Open which will do os.Open (which will be read-only). disk.OpenFile will do os.OpenFile (which can be rw, append) 2015-08-02 17:34:29 +05:30
Anand Babu (AB) Periasamy a728ddc027 probe package to trace & return errors 2015-08-02 02:38:08 -07:00
Harshavardhana aabfd541e1 Merge cmd/donut into minio cmd, deprecate controller RPC request 2015-07-31 12:57:15 -07:00
Harshavardhana 2671b2dbf4 crypto/cleanup: remove unused functions 2015-07-29 13:09:55 -07:00
Krishna Srinivas fdd2c22fa5 Do go fmt on bucket.go 2015-07-29 23:49:25 +05:30
Krishna Srinivas ae8089c9b6 when signature is not available there is no need to compute sha256 2015-07-29 21:09:02 +05:30
Krishna Srinivas bdc00624fd get erros from buffered channel. Return error during object read only if we have readers < k 2015-07-29 19:36:41 +05:30
Harshavardhana d346250f1c Collapse GetPartialObject() into GetObject() 2015-07-28 19:33:56 -07:00
Krishna Srinivas e1280779ed Read from the disks in parallel during object read 2015-07-28 18:04:17 +05:30
Krishna Srinivas 1ea91d2fa2 Use missingEncodedBlocksCount directly instead of "-1" workaround in missingEncodedBlocks[] 
Makes Code more readable
 2015-07-28 16:56:06 +05:30
Harshavardhana b0ea64a04f Strip off quotes from ETag for verifying complete multipart upload 2015-07-25 16:10:05 -07:00
Harshavardhana e082f26e10 Improving EncoderStream to return error only upon non io.EOF. 
io.EOF is okay since io.ReadFull will not have read any bytes at all.

Also making error channel receive only for go routine.
 2015-07-25 15:57:30 -07:00
Krishna Srinivas bcfaa12a4d Encoder now directly reads from the object stream. Using split.Stream() was causing lot of redundant memory operations. 2015-07-26 03:54:39 +05:30
Harshavardhana d6a0e0cc55 Rename more 2015-07-24 18:09:53 -07:00