Krishna Srinivas
0010f0ee10
Remove empty directories while removing an oobject
2015-10-29 23:30:16 -07:00
Harshavardhana
53adfb38f4
Leverage sort Interface to provide sortUnique function
2015-10-23 15:55:41 -07:00
Harshavardhana
dbaa4d8643
Move ListObjects into its own file
2015-10-22 15:39:04 -07:00
Harshavardhana
1f66f4869b
Fix all remaining windows path issues.
2015-10-22 00:05:10 -07:00
Harshavardhana
afa27b9847
On windows translate Prefix, Marker and Delimiter for paths
2015-10-21 22:00:03 -07:00
Harshavardhana
b74852116a
Fix portability issues for arm on raspberry pi
2015-10-20 11:22:00 -07:00
Harshavardhana
dddb1650de
Add 5% cumulative reduction in total size of the disk
...
This is done due to filesystem holding additional metadata and inode space
which is unaccounted for during min-free-disk calculation.
2015-10-19 10:33:34 -07:00
Harshavardhana
179d2d7dac
Add initial cut of auto expiry of objects
2015-10-19 01:34:31 -07:00
Harshavardhana
c065be656c
Implement min-free-disk as a subcommand, deprecate flag
2015-10-19 00:59:20 -07:00
Harshavardhana
5b2fa33bdb
Implementing min-free-disk
2015-10-18 00:23:14 -07:00
Harshavardhana
a8a935f5fd
Improve disk code to return back disk StatFS{} structure
...
```
StatFS {
Total int64
Free int64
FSType string
}
```
Provides more information in a cross platform way.
2015-10-17 20:19:26 -07:00
Harshavardhana
aee0845b2e
Add disk package
...
Implements
- Stat returns total and free disk space supported across platforms
- Type returns type of the filesystem underneath
2015-10-17 16:48:24 -07:00
Harshavardhana
1256ca86d0
Add fs separator
2015-10-17 12:05:12 -07:00
Harshavardhana
2ec679a089
If directory already removed, return nil and move on
2015-10-17 00:13:46 -07:00
Harshavardhana
d534fc5a4f
Implement delete bucket properly with proper error handlings
2015-10-17 00:01:12 -07:00
Harshavardhana
c9af01d807
Enhance listing further, this time handle cases related to common prefixes
2015-10-16 23:11:41 -07:00
Harshavardhana
0eb7f078f9
Implement Bucket ACL support
2015-10-16 19:47:30 -07:00
Harshavardhana
762b798767
Migrate this project to minio micro services code
2015-10-16 11:26:08 -07:00
Harshavardhana
3318cba132
Add windows support for minhttp library
2015-10-11 01:08:16 -07:00
Harshavardhana
7fea9cb550
Add rpc signature handler
2015-10-08 22:28:11 -07:00
Harshavardhana
11048708bb
Implement GetBucketACL - fixes #893
2015-10-08 11:12:44 -07:00
Anand Babu (AB) Periasamy
bf901d3b9a
Merge pull request #895 from abperiasamy/tasker
...
new task model minio server
2015-10-08 02:23:29 -07:00
Anand Babu (AB) Periasamy
b52697e6ad
new task model minio server
2015-10-08 02:20:24 -07:00
Harshavardhana
a060b158c8
Add quick.CheckVersion() to verify config version quickly before unmarshalling the full struct
...
This is needed during migration where we would need to verify the underlying version number
in a quick way.
2015-10-07 17:44:33 -07:00
Harshavardhana
ab5ea997ab
Make erasure Encode and Decode atomic to avoid races
2015-10-06 23:05:01 -07:00
Harshavardhana
1b42398e8b
Canonicalize all the incoming input values, now PresignedPostPolicy works with minio-go
2015-10-06 10:21:28 -07:00
Harshavardhana
d5ce2f6944
Make erasure matrix type not optional choose automatically
...
Remove option of providing Technique and handling errors based on that
choose a matrix type automatically based on number of data blocks.
INTEL recommends on using cauchy for consistent invertible matrices,
while vandermonde is faster we should default to cauchy for large
data blocks.
2015-10-05 22:38:02 -07:00
Harshavardhana
4ed50a8004
Fix encoding bug in donut during encoding phase
...
Stream reading needs to check for length parameter being non zero,
after Reading() a predefined set of buffer length an EOF might be returned
with length == 0.
Erasure taking this zeroed data in might wrongly encode it as part of existing
data blocks which leads to errors while decoding even when the other contents
are intact.
2015-10-05 22:12:53 -07:00
Harshavardhana
f0a8dbecae
Add --json output formatter for server
2015-10-05 00:20:49 -07:00
Harshavardhana
c4faf47e64
First time mode for controller
...
- Upon first time invocation ``minio controller`` would create access keys and secret id
- Upon request passing 'keys' arg ``minio controller`` would provide the keys
- Add colorized notification
2015-10-04 16:42:16 -07:00
Harshavardhana
cfdb29cac0
Simplify signature handling
...
This change brings a new SignatureHandler where Presigned.
Requests without Payload are handled very early before even
going through the call.
This change simplifies Donut codebase to not have signature related
logic for all API's.
Simplification is still needed for Payload based signature eg. PUT/POST calls
, which are still part of the donut codebase, which will be done subsequently
after donut re-write.
2015-10-04 13:15:33 -07:00
Harshavardhana
c8de5bad2f
Implement presigned policy
2015-10-04 00:01:34 -07:00
Anis ELLEUCH
b5ea05d839
A better way to print prettified json syntax error msg
2015-10-03 12:25:44 -07:00
Harshavardhana
3b070dee16
Fix an important metadata getObject bug in donut
2015-10-01 10:18:03 -07:00
Harshavardhana
81cc017f91
Implement presigned signature v4 support
2015-10-01 10:17:47 -07:00
Harshavardhana
50750efb52
Reduce memory usage for memory multipart write by doing io.Pipe() streaming copy
2015-09-30 20:53:30 -07:00
Harshavardhana
8c7c5df770
Prober stringer should avoid frivolous newlines
2015-09-29 10:13:11 -07:00
Harshavardhana
cd52d7a11c
Fix Linux/Mac OS X erasure decoding failure with new Golang version 1.5.1
...
Fixes #871
2015-09-24 21:42:59 -07:00
Harshavardhana
3785489153
Disable multipart for donut backend from being used
...
Will enable it later after cleanup
2015-09-24 18:54:30 -07:00
Harshavardhana
b938e40fb5
Move atomic package to the top and simplify its tests
2015-09-20 13:51:38 -07:00
Harshavardhana
d54488f144
Move all server and controller packages into top-level
2015-09-19 01:07:42 -07:00
Anand Babu (AB) Periasamy
89a86948b5
new version format and some cleanup
2015-09-18 23:33:28 -07:00
Anand Babu (AB) Periasamy
d1f1b7ac31
new version format and some cleanup
2015-09-18 23:27:04 -07:00
Harshavardhana
778f8cd222
Add new rpc tests for Server.Add and Server.List, improve Version.Get RPC to provide more details
2015-09-18 17:44:46 -07:00
Harshavardhana
2a15dd5eab
Enhance signature handler - throw back valid error messages
2015-09-18 15:14:55 -07:00
Harshavardhana
ac93bbb41d
Merge pull request #846 from harshavardhana/new-changes
...
With new auth config changes, restructure the API code to use the new style
2015-09-18 03:45:03 -07:00
Harshavardhana
6a5e5c1826
With new auth config changes, restructure the API code to use the new style
2015-09-18 03:41:05 -07:00
Harshavardhana
b4ce1e8c1d
Add missing reply.Name and add possible failure tests
2015-09-18 03:15:19 -07:00
Harshavardhana
f8bb85aeb7
Enhance auth JSONRPC, now provides persistent output
...
Implements
- Auth.Generate("user")
- Auth.Fetch("user")
- Auth.Reset("user")
This patch also adds testing for each of these cases
2015-09-18 03:02:39 -07:00
Harshavardhana
4bcd86408b
Fix minio header in accordance with rfc2616.txt
2015-09-17 23:46:10 -07:00
Harshavardhana
7093a05ab1
Version is a package now, will be re-used across codebase.
2015-09-17 20:17:33 -07:00
Harshavardhana
1887114444
Fix all the golint complaints about newly added changes
...
Do not use func(this *server), such generic names should not be used
for writing struct methods.
2015-09-17 18:53:42 -07:00
Harshavardhana
03ef6533c8
Add more documentation for probe
2015-09-17 18:10:42 -07:00
Krishna Srinivas
c49407ced4
rename rpc/server.go -> rpc/rpc.go. rpc/server.go will accomodate ServerService
2015-09-17 15:46:51 -07:00
Harshavardhana
3f4b98ca4c
Consolidate controller, move rpc package into controller - remove dangling code in pkg/server
2015-09-15 19:38:36 -07:00
Harshavardhana
8d5f6e0b96
Merge pull request #826 from krishnasrinivas/middleware-cleanup
...
Remove unneeded functions in middleware init
2015-09-15 18:11:45 -07:00
Krishna Srinivas
b1b387b157
Remove unneeded functions in middleware init
2015-09-15 18:09:09 -07:00
Harshavardhana
45d8898019
Merge branch 'parallel-read2' of https://github.com/krishnasrinivas/minio into krishnasrinivas-parallel-read2
...
Make few more changes and rebased with current master
2015-09-15 17:33:33 -07:00
Anand Babu (AB) Periasamy
6930e4d668
logrus logger
2015-09-09 20:28:09 -07:00
Harshavardhana
1e2c010174
Avoid shadowing variables and enable checks to avoid them during build
2015-09-09 15:14:55 -07:00
Harshavardhana
b649eff3fb
Disable logging for now
2015-09-05 21:28:35 -07:00
Harshavardhana
d0f945f8e7
Simplify erasure package for OSX
2015-09-05 20:19:43 -07:00
Harshavardhana
afff3f8885
Revert "Enable controller to have CORS"
...
This reverts commit f39ac24e99
.
2015-08-31 17:15:49 -07:00
Krishna Srinivas
1e82ee1192
Add CORS support to minio s3 server
2015-08-31 16:59:52 -07:00
Harshavardhana
f39ac24e99
Enable controller to have CORS
2015-08-31 01:47:05 -07:00
Harshavardhana
dcf0c71ca3
Merge pull request #810 from harshavardhana/restructure
...
Restructure server code, controller now runs in silo
2015-08-27 17:10:54 -07:00
Harshavardhana
025f95b1d6
Restructure server code, controller now runs in silo
2015-08-27 17:07:32 -07:00
Anand Babu (AB) Periasamy
c11aa1c892
return *probe.Error for Untrace() as well.
2015-08-24 03:35:24 -07:00
Harshavardhana
988d39a5b6
Migrate to golang1.5 release with GO15VENDOREXPERIMENT=1 enabled
2015-08-22 18:35:37 -07:00
Harshavardhana
0e416ea699
Tests were running 4 times due to multiple times the TestingT{} was being called
...
Calling multiple times TestingT{} will hook up runner for Suites for that many times
which would lead to repeated running tests.
Fix it by only initializing it once for all the Suites
2015-08-20 22:32:50 -07:00
Harshavardhana
068d1d1ba9
Add a new quick.Load() function to load directly any config file provided a quick compatible struct{} is also provided
2015-08-20 20:33:49 -07:00
Anand Babu (AB) Periasamy
b49b8cdbe8
return call stack in reverse
2015-08-19 22:40:27 -07:00
Anand Babu (AB) Periasamy
826202716f
skip Env in json printinf if empty
2015-08-19 01:21:28 -07:00
Anand Babu (AB) Periasamy
76c40e075a
make tracePoint public as well
2015-08-19 01:02:39 -07:00
Harshavardhana
ac928b5092
Fix typo in probe
2015-08-18 23:42:41 -07:00
Anand Babu (AB) Periasamy
cdf93e534c
simplify probe APIs
2015-08-18 19:30:17 -07:00
Harshavardhana
e9c5a51bc6
Migrate pkg/quick from mc
2015-08-13 16:29:55 -07:00
Harshavardhana
61175ef091
Migrate to govendor to avoid limitations of godep
...
- over the course of a project history every maintainer needs to update
its dependency packages, the problem essentially with godep is manipulating
GOPATH - this manipulation leads to static objects created at different locations
which end up conflicting with the overall functionality of golang.
This also leads to broken builds. There is no easier way out of this other than
asking developers to do 'godep restore' all the time. Which perhaps as a practice
doesn't sound like a clean solution. On the other hand 'godep restore' has its own
set of problems.
- govendor is a right tool but a stop gap tool until we wait for golangs official
1.5 version which fixes this vendoring issue once and for all.
- govendor provides consistency in terms of how import paths should be handled unlike
manipulation GOPATH.
This has advantages
- no more compiled objects being referenced in GOPATH and build time GOPATH
manging which leads to conflicts.
- proper import paths referencing the exact package a project is dependent on.
govendor is simple and provides the minimal necessary tooling to achieve this.
For now this is the right solution.
2015-08-12 19:24:57 -07:00
Harshavardhana
45b59b8456
Probe revamped to provide for a new WrappedError struct to wrap probes as error interface
...
This convenience was necessary to be used for golang library functions like io.Copy and io.Pipe
where we shouldn't be writing proxies and alternatives returning *probe.Error
This change also brings more changes across code base for clear separation regarding where an error
interface should be passed encapsulating *probe.Error and where it should be used as is.
2015-08-08 00:16:38 -07:00
Harshavardhana
f8141493bd
use command not found helper
2015-08-03 18:06:19 -07:00
Harshavardhana
d09fd8b0a1
Migrate from iodine to probe
2015-08-03 16:33:44 -07:00
Harshavardhana
884e9771b2
More changes to probe to avoid nil dereferences
2015-08-03 01:47:37 -07:00
Harshavardhana
65e4aede82
Minor changes to probe
2015-08-02 20:33:49 -07:00
Anand Babu (AB) Periasamy
697009c0a1
trace on New and add read locks
2015-08-02 11:58:28 -07:00
Harshavardhana
d9493909d8
Merge pull request #788 from krishnasrinivas/open-and-openfile
...
rename disk.OpenFile to Open which will do os.Open (which will be rea…
2015-08-02 11:05:04 -07:00
Krishna Srinivas
ee4432bc40
rename disk.OpenFile to Open which will do os.Open (which will be read-only). disk.OpenFile will do os.OpenFile (which can be rw, append)
2015-08-02 17:34:29 +05:30
Anand Babu (AB) Periasamy
a728ddc027
probe package to trace & return errors
2015-08-02 02:38:08 -07:00
Harshavardhana
aabfd541e1
Merge cmd/donut into minio cmd, deprecate controller RPC request
2015-07-31 12:57:15 -07:00
Harshavardhana
2671b2dbf4
crypto/cleanup: remove unused functions
2015-07-29 13:09:55 -07:00
Krishna Srinivas
fdd2c22fa5
Do go fmt on bucket.go
2015-07-29 23:49:25 +05:30
Krishna Srinivas
ae8089c9b6
when signature is not available there is no need to compute sha256
2015-07-29 21:09:02 +05:30
Krishna Srinivas
bdc00624fd
get erros from buffered channel. Return error during object read only if we have readers < k
2015-07-29 19:36:41 +05:30
Harshavardhana
d346250f1c
Collapse GetPartialObject() into GetObject()
2015-07-28 19:33:56 -07:00
Krishna Srinivas
e1280779ed
Read from the disks in parallel during object read
2015-07-28 18:04:17 +05:30
Krishna Srinivas
1ea91d2fa2
Use missingEncodedBlocksCount directly instead of "-1" workaround in missingEncodedBlocks[]
...
Makes Code more readable
2015-07-28 16:56:06 +05:30
Harshavardhana
b0ea64a04f
Strip off quotes from ETag for verifying complete multipart upload
2015-07-25 16:10:05 -07:00
Harshavardhana
e082f26e10
Improving EncoderStream to return error only upon non io.EOF.
...
io.EOF is okay since io.ReadFull will not have read any bytes at all.
Also making error channel receive only for go routine.
2015-07-25 15:57:30 -07:00
Krishna Srinivas
bcfaa12a4d
Encoder now directly reads from the object stream. Using split.Stream() was causing lot of redundant memory operations.
2015-07-26 03:54:39 +05:30
Harshavardhana
d6a0e0cc55
Rename more
2015-07-24 18:09:53 -07:00
Harshavardhana
63c9cf0c4b
Move from Minimalist Object Storage to Minio Cloud Storage
2015-07-24 17:51:40 -07:00
Harshavardhana
43c908d5b9
ListObjects now considers multipart objects, also move to upstream check.v1
2015-07-18 15:49:41 -07:00
Harshavardhana
966786c78e
remove unnecessary updateAt()
2015-07-17 19:48:09 -07:00
Harshavardhana
c1da10a207
Donut multipart support no get, listobjects() support yet
2015-07-17 16:23:51 -07:00
Harshavardhana
86a887f9d4
Add corresponding tests
2015-07-16 17:40:19 -07:00
Harshavardhana
5eae32f2b0
Return proper InvalidArgument messages like s3 for invalid data for ListObjects(), ListObjectParts(), ListMultipartUploads()
2015-07-16 17:22:45 -07:00
Harshavardhana
e605787e65
Method not allowed is right response for DELETE() operations and add tests
2015-07-16 14:15:24 -07:00
Harshavardhana
1f2e6a40a0
Fix an issue with reusing closed network connetion, changing the way rateLimitedListener is initialized
2015-07-16 10:56:14 -07:00
Harshavardhana
5507a39840
Heal buckets upon init if needed, adding new disks and HUP works
2015-07-16 09:59:05 -07:00
Harshavardhana
e4574c7d6f
Add File() method to Ratelimitlistener for extracting underlying fd()
2015-07-16 00:11:17 -07:00
Harshavardhana
2e8f154f34
Iodine should indent with EmitJSON()
2015-07-16 00:10:55 -07:00
Harshavardhana
4498662c16
Remove scsi non portable code, instead "donut make" implements functionality to instantiate a donut
2015-07-15 11:55:57 -07:00
Harshavardhana
6baf45e360
Fix a crash during listObjects() populating NextMarker
2015-07-15 10:04:15 -07:00
Harshavardhana
2553654e24
Add initial version of heal, remove rebalance
2015-07-14 22:56:41 -07:00
Harshavardhana
e37c5315ec
Live multiple disk removal works properly
2015-07-14 20:46:14 -07:00
Harshavardhana
e1e4908515
Wire up sha512 matching inside donut along with md5sum
2015-07-14 19:47:50 -07:00
Harshavardhana
e885259584
Handle removal of disks - getObject() now reads if disks are missing underneath, add initial stub healing code
2015-07-14 18:53:00 -07:00
Harshavardhana
efbf3eabb7
Return x-amz-request-id for all replies
2015-07-14 14:44:03 -07:00
Harshavardhana
da8b9fd112
Remove global custom config path variables, use get/set methods instead
2015-07-14 11:56:15 -07:00
Harshavardhana
c4cf7635bf
Read req.Body for PutBucket() if any
2015-07-14 09:30:10 -07:00
Harshavardhana
45ddec925c
Fix an ugly multipart bug
2015-07-13 21:38:01 -07:00
Harshavardhana
7ae60a6d10
For missing parts reply back as InvalidPart{}
2015-07-13 19:31:01 -07:00
Harshavardhana
634f70f3b0
WriteObject() encoded data in go routine, fix another multipart issue
2015-07-13 15:56:54 -07:00
Harshavardhana
1bad92356d
Add a ratelimited listener than a ratelimited handler - more precise
2015-07-13 11:12:17 -07:00
Anand Babu (AB) Periasamy
8af5933b07
Merge pull request #743 from flandr/fix-osx
...
Fix OS X build
2015-07-13 10:15:53 -07:00
Nate Rosenblum
ec347f96fd
Fix OS X build
...
- Explicitly cast statfs_t members to int64 (this structure is
platform-specific)
- Add pass-through New methods to Darwin SHA package
- Move scsi pkg types to common translation unit (package was empty)
- Add stub implementations mount/disk ops for OS X
2015-07-13 10:06:55 -07:00
Harshavardhana
55e4d0c6a5
mkdonut now creates a donut processing cli args
2015-07-12 21:21:31 -07:00
Harshavardhana
58a1d865a9
Renaming nimble to minhttp
2015-07-12 18:01:52 -07:00
Harshavardhana
847440196e
Add signature v4 tests
2015-07-12 13:36:02 -07:00
Harshavardhana
7615a6bfe5
HEAD shouldn't have any body, handle it in writeErrorResponse()
2015-07-11 10:34:55 -07:00
Harshavardhana
8977f9a524
cached api test should also have a custom config path, would conflict with your .minio local path
2015-07-10 20:39:30 -07:00
Harshavardhana
97d4a27c7e
Do not reply on ignoredHeaders for server, rely on SignedHeaders sent as part of Authorization header
2015-07-10 19:37:22 -07:00
Harshavardhana
53f5d2c32b
nodejs http library sends Connection header during HTTP request, this clobbers up the signature handling ignore it
2015-07-10 15:44:47 -07:00
Harshavardhana
29838bb851
Cleanup temporary writers upon errors during putObject(), all metadata() write operations
2015-07-10 14:11:04 -07:00
Harshavardhana
15dd0df187
Support signature v4 at rest
2015-07-10 11:49:27 -07:00
Harshavardhana
7fa514351c
Rename definitions to log.go, add valid prefixes
2015-07-10 11:11:20 -07:00
Harshavardhana
d5ffc16f25
Add abbreviated close response, to avoid any leaks
2015-07-10 10:20:00 -07:00
Harshavardhana
e5006c738d
Handle both space and non-space characters, in signature v4 - add errors for all API's
2015-07-09 21:49:58 -07:00
Harshavardhana
84f427f14a
All other API's now support signature v4
2015-07-09 19:45:56 -07:00
Harshavardhana
00890c254e
CompleteMultipartUpload and CreateObjectPart now fully support signature v4
2015-07-09 19:01:15 -07:00
Harshavardhana
89c1215194
PutObject handler gets initial support for signature v4, working
2015-07-09 16:44:38 -07:00
Harshavardhana
d461fa5ab1
Add mountinfo functions for detecting mount disks, and other rpc changes
2015-07-09 12:25:29 -07:00
Harshavardhana
8654ddb566
Generate auth now saves in ${HOME}/.minio/users.json, also authHandler verifies request validity
2015-07-08 21:53:13 -07:00
Harshavardhana
375860077d
HTTP header Content-Length signifies body length of the request, if its smaller reply appropriately
...
This patch also handles large individual part sizes > 5MB by using less memory copies.
2015-07-08 20:56:41 -07:00
Harshavardhana
ec33d79d57
Add server side signaturev4 check, not wired up to the readers yet.
2015-07-08 16:57:03 -07:00
Harshavardhana
396b728031
Add auth rpc service to generate access keys, add corresponding test
2015-07-08 14:40:39 -07:00
Harshavardhana
770fd23afa
Renaming keys as auth, working towards signature v4 support for all put objects
2015-07-08 14:17:16 -07:00
Harshavardhana
2413a110e6
Fix SSL support, pointer indirection caused nil buffers
2015-07-08 11:02:15 -07:00
Harshavardhana
d1deda3a96
Add API tests for both donut on disk and donut cache
2015-07-07 19:39:46 -07:00
Harshavardhana
ece797c16e
Add rpc tests
2015-07-07 17:27:34 -07:00
Harshavardhana
676b9058de
Separate out memory statistics and system information into two different services
2015-07-07 16:59:20 -07:00
Harshavardhana
8abb96c030
If NodeDisks are not empty do not impose cache maxSize restriction
2015-07-07 16:41:40 -07:00
Harshavardhana
a50a44b0ca
Add nimbleNet tests
2015-07-07 16:15:47 -07:00
Harshavardhana
317096a0c4
Add net.Addr wrapper with IsEqual() and use it.
2015-07-07 15:33:08 -07:00
Harshavardhana
11b893804c
Moving os.MkdirAll() inside atomic for auto parent directory creates
2015-07-07 12:35:57 -07:00
Harshavardhana
52cd23ad9f
Move atomic file writes into its own package, use them inside quick and disk packages
2015-07-07 12:29:14 -07:00
Harshavardhana
3622fbc87d
Across donut, split, nimble some code cleanup
2015-07-06 21:55:21 -07:00
Harshavardhana
bbb89b5776
Add multi-thread protection and also allow atomic file creates, rename upon Close()
2015-07-06 18:54:32 -07:00
Harshavardhana
c2c7bdf0cd
Cleanup nimble http
2015-07-06 18:51:20 -07:00
Harshavardhana
b029d0a5f0
Avoid config reload all the time, reload is manually triggerred from outside
2015-07-06 17:26:35 -07:00
Harshavardhana
8b94c53345
Fix issues with multipart upload
2015-07-06 16:22:27 -07:00
Harshavardhana
474954022e
Add modified grace library from facebookgo, rename it as nimble
2015-07-06 15:40:12 -07:00
Harshavardhana
1d64e4b6c1
Add Donut rpc service for sending changes to configuration files
2015-07-06 11:10:06 -07:00
Harshavardhana
10b082144e
Add updateConfig code to load config changes if possible for every function
2015-07-05 22:46:42 -07:00
Harshavardhana
ba0a5ed416
Add sighup, sigusr2 into trapping code, to trap signals for reloading configuration.
...
Need to still figure out a way of graceful restarts - gave facebookgo/httpdown a shot,
but it is not suitable.
2015-07-05 21:40:53 -07:00
Harshavardhana
4a27ab0e58
Fix another deadlock inside CreateObjectPart() code, premature return without Unlocking()
...
Also this patch changes the cache key element to be interface{} type not string.
2015-07-05 20:26:32 -07:00
Harshavardhana
75788c7a1d
Fix go installation check on amazon instance
2015-07-05 18:12:58 -07:00
Harshavardhana
7f0c14f2b7
Add basic controller code, initiating json rpc connection getting list of disks and memstats for now.
2015-07-05 17:17:41 -07:00
Harshavardhana
a3ccb9d405
Rename stuttered service names and make them appropriate
2015-07-05 10:19:23 -07:00
Harshavardhana
e66a84242a
Add disk detection for Linux, add new RPC service GetDiskInfoService(), remove dummy HelloService()
2015-07-05 02:08:33 -07:00
Harshavardhana
bab4a47525
Move to container/list datastructure from map[string][]byte
2015-07-04 17:08:23 -07:00
Harshavardhana
aa67a19e99
Implement new CPU detection using cpuid, cpuidex plan9 instructions from klauspost/cpuid project, remove C code
2015-07-04 14:28:16 -07:00
Harshavardhana
0a827305ad
Implement metadata cache, metadata cache is used by top level donut right now. Rename trove as data cache
...
We should use it internally everywhere.
2015-07-03 21:09:57 -07:00
Harshavardhana
14844f48dd
Make donut do everything as an atomic operation, this avoids all the deadlocks and races
2015-07-03 17:16:58 -07:00
Harshavardhana
30fc14e703
Restructure codebase move crypto, checksum to top-level, move `split
` into donut, move crypto/keys into api/auth
2015-07-03 15:24:51 -07:00
Harshavardhana
8a4e7bcdcf
Add full API tests, move storage/donut to donut, add disk tests as well
2015-07-03 14:36:29 -07:00
Harshavardhana
7c37e9d06a
Make donut fully integrated back into API handlers
2015-07-02 21:04:04 -07:00
Harshavardhana
12bde7df30
Add simple Ticket Master which pro-actively sends messages on proceedChannel
...
Handlers are going to wait on proceedChannel, this the initial step towards
providing priority for different set of API operations
2015-07-02 21:04:04 -07:00
Harshavardhana
5cfb05465e
Add cache, donut tests separately - fix behavior differences
...
Remove priority queue, implement it using a simpler channels
2015-07-02 21:04:04 -07:00
Harshavardhana
ebe61d99d9
Use cache Append() for saving objects in memory, GetObject() caches un-cached entries while reading
2015-07-02 21:04:04 -07:00
Harshavardhana
bce93c1b3a
Integrate cache with donut, add tests
2015-07-02 21:04:04 -07:00
Harshavardhana
0533abf6a8
Make priority queue lambda function return error over a channel
2015-07-02 21:04:04 -07:00
Harshavardhana
38a6ce36e5
Remove slow AppendUniq code, rolling through over a slice is in-efficient
...
Remove it and use map instead
2015-07-02 21:04:04 -07:00
Harshavardhana
84810162f5
Add simple Version and GetSysInfo services
2015-07-02 21:04:04 -07:00
Harshavardhana
14ec42d646
Add initial implementation of priority queue, uses container/heap
2015-07-02 21:04:04 -07:00
Harshavardhana
eb5aa19dfa
Remove custom Config, will use quick Config instead for user access keys
2015-07-02 21:04:04 -07:00
Harshavardhana
701c3e5242
Add new RPC helpers wrapping over regular rpc packages, add middleware chaining ability
2015-07-02 21:04:04 -07:00
Harshavardhana
4addf7a996
Restructure API handlers, add JSON RPC simple HelloService right now.
2015-07-02 21:04:04 -07:00
Harshavardhana
335c7827eb
More donut, cache, api cleanup
2015-07-02 21:04:04 -07:00
Harshavardhana
dc0df3dc0e
Breakaway from driver model, move cache into donut
2015-07-02 21:04:03 -07:00
Harshavardhana
72572d6c71
Remove some api server code bringing in new cleanup
2015-07-02 21:04:03 -07:00
Harshavardhana
c2031ca066
Add server and control command
2015-07-02 21:04:03 -07:00
Frederick F. Kautz IV
cfbc169034
Fixing API definitions
2015-07-02 13:14:21 -07:00
Harshavardhana
8f61d6b6be
Move memory code out, add it as layer on top of existing donut code
...
Just like how http.Handlers can be overlayed on top of each other
with each implementing ServeHTTP().
drivers.Driver can be overlayed on top of each other in similar manner
which would implement the drivers.Driver interface.
API <----> cache <----> donut <----> donut(format)
2015-06-30 10:09:12 -07:00
Harshavardhana
ab6e16bb41
Add dummy driver for community to submit new drivers
2015-06-29 16:43:50 -07:00
Harshavardhana
12de98fb62
Rename memory driver as cache
2015-06-29 16:43:50 -07:00
Harshavardhana
2571342451
Filesystem goes the high road *again*
2015-06-29 16:43:42 -07:00
Harshavardhana
f74d6138da
IsValidBucket() is sufficient we don't need to verify for "."
2015-06-29 15:15:54 -07:00
Harshavardhana
3109909355
Handle couple of cases of OOM conditions, move caching to GetObject() rather than PutObject()
2015-06-29 12:28:50 -07:00
Harshavardhana
d07d0c670a
Return back proper errors in writeObjectData(), rename few functions
2015-06-29 11:46:35 -07:00
Harshavardhana
10c807f233
Put object on successful write returns full metadata, to avoid subsequent GetObjectMetadata() calls in driver
2015-06-29 11:15:46 -07:00