Harshavardhana
474954022e
Add modified grace library from facebookgo, rename it as nimble
2015-07-06 15:40:12 -07:00
Harshavardhana
1d64e4b6c1
Add Donut rpc service for sending changes to configuration files
2015-07-06 11:10:06 -07:00
Harshavardhana
10b082144e
Add updateConfig code to load config changes if possible for every function
2015-07-05 22:46:42 -07:00
Harshavardhana
ba0a5ed416
Add sighup, sigusr2 into trapping code, to trap signals for reloading configuration.
...
Need to still figure out a way of graceful restarts - gave facebookgo/httpdown a shot,
but it is not suitable.
2015-07-05 21:40:53 -07:00
Harshavardhana
4a27ab0e58
Fix another deadlock inside CreateObjectPart() code, premature return without Unlocking()
...
Also this patch changes the cache key element to be interface{} type not string.
2015-07-05 20:26:32 -07:00
Harshavardhana
75788c7a1d
Fix go installation check on amazon instance
2015-07-05 18:12:58 -07:00
Harshavardhana
7f0c14f2b7
Add basic controller code, initiating json rpc connection getting list of disks and memstats for now.
2015-07-05 17:17:41 -07:00
Harshavardhana
a3ccb9d405
Rename stuttered service names and make them appropriate
2015-07-05 10:19:23 -07:00
Harshavardhana
e66a84242a
Add disk detection for Linux, add new RPC service GetDiskInfoService(), remove dummy HelloService()
2015-07-05 02:08:33 -07:00
Harshavardhana
bab4a47525
Move to container/list datastructure from map[string][]byte
2015-07-04 17:08:23 -07:00
Harshavardhana
aa67a19e99
Implement new CPU detection using cpuid, cpuidex plan9 instructions from klauspost/cpuid project, remove C code
2015-07-04 14:28:16 -07:00
Harshavardhana
0a827305ad
Implement metadata cache, metadata cache is used by top level donut right now. Rename trove as data cache
...
We should use it internally everywhere.
2015-07-03 21:09:57 -07:00
Harshavardhana
14844f48dd
Make donut do everything as an atomic operation, this avoids all the deadlocks and races
2015-07-03 17:16:58 -07:00
Harshavardhana
30fc14e703
Restructure codebase move crypto, checksum to top-level, move `split
` into donut, move crypto/keys into api/auth
2015-07-03 15:24:51 -07:00
Harshavardhana
8a4e7bcdcf
Add full API tests, move storage/donut to donut, add disk tests as well
2015-07-03 14:36:29 -07:00
Harshavardhana
7c37e9d06a
Make donut fully integrated back into API handlers
2015-07-02 21:04:04 -07:00
Harshavardhana
12bde7df30
Add simple Ticket Master which pro-actively sends messages on proceedChannel
...
Handlers are going to wait on proceedChannel, this the initial step towards
providing priority for different set of API operations
2015-07-02 21:04:04 -07:00
Harshavardhana
5cfb05465e
Add cache, donut tests separately - fix behavior differences
...
Remove priority queue, implement it using a simpler channels
2015-07-02 21:04:04 -07:00
Harshavardhana
ebe61d99d9
Use cache Append() for saving objects in memory, GetObject() caches un-cached entries while reading
2015-07-02 21:04:04 -07:00
Harshavardhana
bce93c1b3a
Integrate cache with donut, add tests
2015-07-02 21:04:04 -07:00
Harshavardhana
0533abf6a8
Make priority queue lambda function return error over a channel
2015-07-02 21:04:04 -07:00
Harshavardhana
38a6ce36e5
Remove slow AppendUniq code, rolling through over a slice is in-efficient
...
Remove it and use map instead
2015-07-02 21:04:04 -07:00
Harshavardhana
84810162f5
Add simple Version and GetSysInfo services
2015-07-02 21:04:04 -07:00
Harshavardhana
14ec42d646
Add initial implementation of priority queue, uses container/heap
2015-07-02 21:04:04 -07:00
Harshavardhana
eb5aa19dfa
Remove custom Config, will use quick Config instead for user access keys
2015-07-02 21:04:04 -07:00
Harshavardhana
701c3e5242
Add new RPC helpers wrapping over regular rpc packages, add middleware chaining ability
2015-07-02 21:04:04 -07:00
Harshavardhana
4addf7a996
Restructure API handlers, add JSON RPC simple HelloService right now.
2015-07-02 21:04:04 -07:00
Harshavardhana
335c7827eb
More donut, cache, api cleanup
2015-07-02 21:04:04 -07:00
Harshavardhana
dc0df3dc0e
Breakaway from driver model, move cache into donut
2015-07-02 21:04:03 -07:00
Harshavardhana
72572d6c71
Remove some api server code bringing in new cleanup
2015-07-02 21:04:03 -07:00
Harshavardhana
c2031ca066
Add server and control command
2015-07-02 21:04:03 -07:00
Frederick F. Kautz IV
cfbc169034
Fixing API definitions
2015-07-02 13:14:21 -07:00
Harshavardhana
8f61d6b6be
Move memory code out, add it as layer on top of existing donut code
...
Just like how http.Handlers can be overlayed on top of each other
with each implementing ServeHTTP().
drivers.Driver can be overlayed on top of each other in similar manner
which would implement the drivers.Driver interface.
API <----> cache <----> donut <----> donut(format)
2015-06-30 10:09:12 -07:00
Harshavardhana
ab6e16bb41
Add dummy driver for community to submit new drivers
2015-06-29 16:43:50 -07:00
Harshavardhana
12de98fb62
Rename memory driver as cache
2015-06-29 16:43:50 -07:00
Harshavardhana
2571342451
Filesystem goes the high road *again*
2015-06-29 16:43:42 -07:00
Harshavardhana
f74d6138da
IsValidBucket() is sufficient we don't need to verify for "."
2015-06-29 15:15:54 -07:00
Harshavardhana
3109909355
Handle couple of cases of OOM conditions, move caching to GetObject() rather than PutObject()
2015-06-29 12:28:50 -07:00
Harshavardhana
d07d0c670a
Return back proper errors in writeObjectData(), rename few functions
2015-06-29 11:46:35 -07:00
Harshavardhana
10c807f233
Put object on successful write returns full metadata, to avoid subsequent GetObjectMetadata() calls in driver
2015-06-29 11:15:46 -07:00
Harshavardhana
6921328b93
Avoid frivolous GetObjectMetadata() calls at driver level, return back all the information in donut ListObjects()
2015-06-29 11:14:58 -07:00
Harshavardhana
d8f7896a43
Expand http server struct to store more values
2015-06-29 00:12:28 -07:00
Harshavardhana
42c0287943
Use errorChannels only for services not for drivers, reduce them to use simple functions
2015-06-28 23:59:47 -07:00
Harshavardhana
91e5f648cb
Move to Set() not Append() due to large memory reference copy
2015-06-28 10:13:12 -07:00
Harshavardhana
ac4d8fe478
Add free() method for proxyReader to aggressively de-allocate Read data, to handle certain out of memory conditions
...
There are still some more out there
2015-06-27 20:43:25 -07:00
Harshavardhana
05f8654e3d
Add Append() method to trove cache for appending data to an existing key
...
This largely avoids a large buffer copy which would accumulate inside proxyReader{}
This patch also implements "initialize()" function to init and populate data
on all the existing buckets, avoiding the redundant ListBuckets() invoked by
every API call.
2015-06-27 20:25:24 -07:00
Harshavardhana
367772b988
Make sure to populate on disk data into memory upon first API requests
2015-06-27 18:25:21 -07:00
Harshavardhana
45a7eab804
An attempt to bring in memory layer into donut driver
2015-06-27 17:23:34 -07:00
Harshavardhana
f3c25bcfc4
Keeping the lexical order same add optimizations, provide a comprehensive response from ListObjects()
2015-06-27 13:12:44 -07:00
Harshavardhana
ae66ae42c4
Rename functions for their purpose
2015-06-27 12:39:11 -07:00
Harshavardhana
9a4680475f
Object metadata was wrongly misconstrued to be mutable, handle it
2015-06-26 23:22:53 -07:00
Harshavardhana
3aa6d90c5e
Avoid racy maps, read from disk on success return quickly. Many more optimizations
2015-06-26 19:49:37 -07:00
Harshavardhana
68974918ac
Donut moves to typed metadata, removing the necessity for strconv, parsing and string converstions
2015-06-26 16:23:12 -07:00
Harshavardhana
767d3743ee
Add new metadata definitions and use them
2015-06-26 13:34:09 -07:00
Harshavardhana
9958e34772
Import quick key value store from Minio Client for persistent state files, primarily for donut
2015-06-25 19:57:31 -07:00
Harshavardhana
fb9adb5524
Donut cleanup another set
...
- Make sure to close all readers
- Fix errors in api_testsuite c.Assert(err, IsNil) should be done right after each function call
2015-06-25 18:54:34 -07:00
Harshavardhana
eec66f195a
Take all the ListObjects into bucket handlers
...
Earlier the listing would wait for all the objects to be processed
this is essentially very time consuming considering even for 100,000
files.
2015-06-25 18:04:29 -07:00
Harshavardhana
45e9d25931
Remove more bloated code - simplify
2015-06-25 13:02:08 -07:00
Harshavardhana
82dcbf262d
Add simple locking for donut API for now - fixes #671
2015-06-25 11:29:11 -07:00
Harshavardhana
03b4d3b308
Go vet fixes for donut
2015-06-24 21:07:03 -07:00
Harshavardhana
57a2b53178
Removing further bloated code simplifying
2015-06-24 21:03:39 -07:00
Harshavardhana
a2c205ff2e
Use external package disk for donut.
2015-06-24 21:03:02 -07:00
Harshavardhana
841ff01412
Move disk into its own package, remove bloated code
2015-06-24 20:13:47 -07:00
Harshavardhana
1682c748ac
Remove unnecessary interfaces from donut, cleanup
2015-06-24 19:43:38 -07:00
Harshavardhana
3498872467
Add sha256 and sha512 windows compatibility layer
2015-06-24 14:39:41 -07:00
Harshavardhana
77d35b87d4
Fix a bug on windows regarding blockSSE3 calculation
2015-06-24 14:24:33 -07:00
Harshavardhana
f1410731db
Add windows code for sha1 and crc32c
2015-06-24 14:16:32 -07:00
Harshavardhana
ba2d3dea74
Fix wrong TMPFS listing in donut filesystem map
2015-06-23 23:16:06 -07:00
Harshavardhana
e818bc7187
Trim iodine path properly, so that now errors have github.com/minio/minio prefixed
2015-06-23 13:36:54 -07:00
Harshavardhana
2fd52ca284
Add missing strongly typed errors for Donut
2015-06-23 11:54:44 -07:00
Harshavardhana
7a060110ff
Fix BUILDDEPS paths for golang installation
2015-06-22 12:07:23 -07:00
Harshavardhana
3bf64f5669
Verify d.donut to be non-nil, usually happens when multiple go versions compilations are linked - possible cause for #659
2015-06-20 11:03:17 -07:00
Harshavardhana
641f07cecf
Use filepath everywhere instead of path.{} functions for portability - fixes #656
2015-06-18 16:02:45 -07:00
Harshavardhana
573a6134b2
Remove redundant !ok for map
2015-06-17 22:36:46 -07:00
Harshavardhana
e9a3fd677a
Hold lock on GetGlobalStateKey() - fixes #652
2015-06-17 20:37:49 -07:00
Harshavardhana
b793f53d48
Minior change filter() to filterObjects()
2015-06-17 20:35:44 -07:00
Harshavardhana
d4e71beb54
Add time.Since on buckets created time to avoid deleting buckets spuriously during eviction of objects
2015-06-13 11:28:04 -07:00
Harshavardhana
636788997b
Minor optimization avoiding one unnecessary copy
2015-06-11 14:23:05 -07:00
Harshavardhana
6d44274db6
Make sure to reply back ETag with quotes for s3 compliance
2015-06-09 01:13:02 -07:00
Harshavardhana
a947767339
Remove unnecessary tlsHelpers, SSL works with domain certificate
2015-06-08 20:54:41 -07:00
Harshavardhana
c6ecd94420
Enable TLS and use it
2015-06-08 20:10:59 -07:00
Harshavardhana
64b014369c
Return back entity too large for createObject requests bigger than cache size
2015-06-08 17:10:07 -07:00
Harshavardhana
ab1976bc14
Load from persisted multipart session, now multipart uploads can resume even after restarting minio fs server
2015-06-08 15:15:37 -07:00
Harshavardhana
2976f109df
More multipart fixes, filter out parts and multipart metadata from listObjects
2015-06-08 14:08:36 -07:00
Harshavardhana
e08d59be00
Make sure to delete uploadid's from active session file properly
2015-06-08 12:25:51 -07:00
Harshavardhana
0d3710cb9b
Store in json file instead of gob for fs metadata
2015-06-08 12:08:01 -07:00
Harshavardhana
8efc842b59
Add delete handlers and reply back as 'NotImplemented' instead of 404
2015-06-08 11:06:06 -07:00
Harshavardhana
8a320c4447
Enhance logging handler to print in humanize form for ContentLength
2015-06-08 02:14:09 -07:00
Harshavardhana
cefd740002
Put back the older conn_limit
2015-06-06 18:11:38 -07:00
Harshavardhana
371651bde6
Add rate limiter instead of connection limit for now
2015-06-06 18:04:55 -07:00
Harshavardhana
21916dc0a1
Fix a deadlock in trove when evicting objects when we reach the maxSize used
2015-06-06 16:02:52 -07:00
Harshavardhana
8b43655d9f
Make connection limit configurable
2015-06-06 14:31:50 -07:00
Harshavardhana
a344f5b34d
Misc changes before capturing proper video
...
- Disable management UI command option
- Remove featureflags option, filesystem and memory already support multipart
- Print informative messages after starting minio server
2015-06-03 18:27:15 -07:00
Harshavardhana
097f70a3c5
Enable filesystem for API tests, this patch also implements ListMultipartUploads()
2015-06-01 20:57:13 -07:00
Harshavardhana
bfb9209689
Skip files and weird directories from listBuckets(), allowing this so that minio fs mode can be used as ftp service
2015-06-01 16:43:43 -07:00
Harshavardhana
d86ba5f336
Implement multipart support
...
- implemented NewMultipartUpload()
- implemented CreateObjectPart()
- implemented ListObjectParts()
- implemented AbortMultipartUpload()
- implemented CompleteMultipartUpload()
2015-06-01 16:25:11 -07:00
Harshavardhana
c07d48e7e8
ErrorResponse from minio server needed proper xml translations
2015-06-01 12:19:27 -07:00
Harshavardhana
a91e519692
Make caching a package trove, and use it inside memory driver
2015-05-31 18:41:56 -07:00
Anis Elleuch
7e5a26f42c
Protect the exported method Delete()
2015-05-31 20:29:42 +01:00
Harshavardhana
82a0eac659
Add filesystem factory functions and add related cli options
2015-05-30 03:23:06 -07:00
Harshavardhana
0cc63706bb
Resurrect back fileystem driver
2015-05-30 03:12:01 -07:00
Anis Elleuch
d360d0fec3
Forbid the upload of files bigger than the memory backend capacity
2015-05-28 22:07:02 +01:00
Harshavardhana
554a244b47
Add donut OSX support
2015-05-26 02:14:34 -07:00
Harshavardhana
e46660a966
Use http.TimeFormat to return, S3 expects us to return in GMT not UTC
2015-05-25 20:17:04 -07:00
Harshavardhana
19250296c6
Handle authorization header better
2015-05-24 20:59:53 -07:00
Harshavardhana
9666f2e5bf
Relax minium object size limit, one can really upload a 0byte file
...
For example
~~~
$ mc ls https://s3.amazonaws.com/ferenginar/test.go
[2015-05-23 12:02:06 PDT] 0B test.go
$ curl -i -X HEAD https://s3.amazonaws.com/ferenginar/test.go
HTTP/1.1 200 OK
x-amz-id-2: ZMWLriPH+uQJ8IsaMHmuNOU/FXvdSq+s6O7ugUI9hZ695XJTNAZ1utKxh03w5Jcf
x-amz-request-id: F5A11F533B74DD8F
Date: Sat, 23 May 2015 19:19:29 GMT
Last-Modified: Sat, 23 May 2015 19:02:06 GMT
ETag: "d41d8cd98f00b204e9800998ecf8427e"
Accept-Ranges: bytes
Content-Type: binary/octet-stream
Content-Length: 0
Server: AmazonS3
~~~
2015-05-23 12:19:16 -07:00
Harshavardhana
e11f9110b6
add missing validate access keys after being extracted from AuthHeader
2015-05-21 19:23:47 -07:00
Harshavardhana
ad60b97d52
Fix some donut TODO's regarding bucket created date
2015-05-21 11:13:49 -07:00
Harshavardhana
cd817d34df
Finish last bits of api testsuite on server
2015-05-21 02:57:03 -07:00
Harshavardhana
f00bfd38da
Bump gocyclo back to 16
2015-05-19 18:32:50 -07:00
Harshavardhana
3d61ba5930
Fix duplicate map allocation
2015-05-18 13:14:01 -07:00
Harshavardhana
e4c0d574dc
Update totalnumber of multiparts inside CreateObjectPart(), also introduce support for UploadIDMarker and Prefix for listMultipartUploads
2015-05-18 12:24:36 -07:00
Harshavardhana
7ce47130fd
HEAD request should have Content-Length for only successful response, there is no response body for errors, just header is sufficient - fixes #603
...
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.13
""" in the case of the HEAD method, the size of the entity-body that would have been sent had the request been a GET. """
2015-05-15 21:55:59 -07:00
Harshavardhana
317637fcb8
Remove redundant WriteHeader calls
2015-05-15 20:58:28 -07:00
Harshavardhana
5c5761ae16
Remove unnecessary GetObjectMetadata prefix argument
2015-05-15 19:25:17 -07:00
Harshavardhana
6275238a47
For multiparts use its own cache avoids deadlocks during object expiration
...
Using same object cache as regular objects for multiparts causes deadlocks during object expiration routine.
As a side affect this also fixes the bug of listing multiparts during regular 'listing' of objects.
2015-05-14 20:26:59 -07:00
Harshavardhana
c41ebf7535
Fixing a race in NewMultipartUpload() session for simultaneous requests, observed at play.minio.io
2015-05-14 19:11:19 -07:00
Harshavardhana
8fb427a892
Add more tests for multipart now in driver api_testsuite
2015-05-14 17:28:39 -07:00
Harshavardhana
dba7c94571
Add tests for abortMultipartUpload, listobjectparts, listmultipartuploads
2015-05-14 17:07:30 -07:00
Harshavardhana
1bd94ec6ab
An attempt to implement ListMultipartUploads()
2015-05-14 16:22:47 -07:00
Harshavardhana
82b043005a
Avoid one last memory copy
2015-05-14 11:26:55 -07:00
Harshavardhana
5498c90788
Add a new validateContentTypeHandler{}, verify Accept header earlier
2015-05-13 12:22:15 -07:00
Harshavardhana
de56909d55
More scripts and code from Minio-io to minio
2015-05-11 16:43:38 -07:00
Harshavardhana
8d06504068
Change all minio-io path to minio
2015-05-11 16:23:10 -07:00
Harshavardhana
e39de4f603
Fix typo introduced in previous commit
2015-05-11 14:24:31 -07:00
Harshavardhana
f45635524f
Remove SignatureV2 support, bring in SignatureV4 header only validation for now
2015-05-11 13:38:40 -07:00
Harshavardhana
b48b7f51ef
Use convenience functions for url.Values over looping through keys, fix a bug in listObjectParts()
2015-05-10 11:32:41 -07:00
Harshavardhana
14b5828505
Handle partNumberMarker with listObjectParts now and other fixes
2015-05-09 19:39:00 -07:00
Frederick F. Kautz IV
cf559fa48e
Limiting upload id size
2015-05-09 18:28:50 -07:00
Frederick F. Kautz IV
61a9ea0f4b
Adding support for memory driver test
2015-05-09 18:15:27 -07:00
Frederick F. Kautz IV
058604aa21
Adding tests and fixes for multipart uploads uncovered from tests
2015-05-09 17:42:14 -07:00
Harshavardhana
8860aa0c8c
Implement AbortMultipart
2015-05-09 16:09:50 -07:00
Harshavardhana
18c7f620cc
Add listparts support
2015-05-09 13:55:55 -07:00
Harshavardhana
04e388d620
Handle two different styles of time format, s3cmd now compatible
2015-05-08 16:01:59 -07:00
Harshavardhana
01b474fca3
Move expiration code into its own and add ticker select case
2015-05-08 12:37:49 -07:00
Harshavardhana
23a545673d
Verify incoming md5sum for invidual parts in complete multipart request and some simplification of the code
...
Add two new functions for session cleanup
2015-05-08 03:59:05 -07:00
Harshavardhana
616241a2c1
Re-enable bucket deletion, this time with uploadID sitting there
2015-05-08 02:28:38 -07:00
Harshavardhana
2ea10c798b
Implement proper errors for Multipart
2015-05-08 02:02:51 -07:00
Harshavardhana
82c3656f79
Reply back CompleteMultipartUploadResult properly with final ETag computed
...
- Now s3 libraries and also objectstorage-go work properly
2015-05-07 23:01:20 -07:00
Frederick F. Kautz IV
ae5c99d7bb
Router should work with ?uploads unescaped now
2015-05-07 21:37:34 -07:00
Frederick F. Kautz IV
9dc7b82b39
Merge pull request #576 from fkautz/pr_out_adding_multipart_support
2015-05-07 20:00:20 -07:00
Frederick F. Kautz IV
f050d5e974
Adding multipart support
2015-05-07 19:55:30 -07:00
Frederick F. Kautz IV
ed71c28c37
Fixing panic in intelligent, attempted to delete nonexistent key
2015-05-07 19:12:49 -07:00
Frederick F. Kautz IV
d0c4334834
Adding feature flags
2015-05-07 13:56:54 -07:00
Harshavardhana
1df28f3060
Get rid of sub-domain code and some other formatting changes
2015-05-07 00:21:40 -07:00
Harshavardhana
6d3d92694f
Add Marker support for donut, also add a testing code
2015-05-05 22:18:24 -07:00
Harshavardhana
75028c2ad1
Add lexicographic Marker/NextMarker support for recursive listing of objects.
...
Also update times when an object is accessed logic
2015-05-05 18:05:34 -07:00
Frederick F. Kautz IV
24af2222c9
Adding xmlns
2015-05-05 12:54:34 -07:00
Harshavardhana
193a6606db
Fix a potential race between ExpireObjects() and Set() over items map
2015-05-04 22:36:47 -07:00
Harshavardhana
ed1259d6f0
Deprecate LRU use just map[string]interface{} and call it intelligent 'cache'
2015-05-04 22:01:56 -07:00
Harshavardhana
63edb1e9a0
Nullify list memory to nil as we remove the element
2015-05-04 17:22:08 -07:00
Harshavardhana
d63064b8af
Optimize memory usage in GetPartialObject() for memory driver
2015-05-04 15:26:56 -07:00
Harshavardhana
c8f31d97a8
Modify LRU further to add object expiration
2015-05-04 04:32:44 -07:00
Harshavardhana
670f997b07
Bring blockingWriter from client and use it here
2015-05-04 03:55:43 -07:00
Harshavardhana
d0df548eb5
Use new LRU inside memory driver
2015-05-04 03:55:43 -07:00
Harshavardhana
f7caef2d26
Change CreateObject() to take size argument from content-length
2015-05-04 03:55:43 -07:00
Harshavardhana
c342ce1588
New LRU based on GroupCache which keeps track of memory rather than entries
2015-05-04 03:55:19 -07:00
Frederick F. Kautz IV
3cf045fd1b
Deleting key from lastAccessedObjects on eviction
2015-05-01 22:01:07 -07:00
Harshavardhana
9ec8925f1c
Rename IsValidObject() to IsValidObjectName()
2015-05-01 20:31:26 -07:00
Frederick F. Kautz IV
33ecba8e49
Reimagining memory settings cli input
2015-05-01 18:12:58 -07:00
Frederick F. Kautz IV
25c268ac88
Attempt to fix memory lock on Get preventing write
2015-05-01 17:45:40 -07:00
Frederick F. Kautz IV
85f1b46492
Removing log message from test
2015-05-01 17:32:18 -07:00
Frederick F. Kautz IV
7cf86e6114
Fix docker support
2015-05-01 15:23:39 -07:00
Harshavardhana
2c7b30e5e1
Even mux vars don't help, handle it by looking at req.URL.Path
2015-05-01 12:41:17 -07:00
Frederick F. Kautz IV
b86b26e7e5
Exposing expiration for memory driver
2015-04-30 21:20:54 -07:00
Harshavardhana
c01fb92eca
Merge pull request #557 from harshavardhana/pr_out_make_builddate_universal
2015-04-30 21:19:04 -07:00
Harshavardhana
bbc32d6ac0
Disable domain based routing for now
2015-04-30 21:15:53 -07:00
Harshavardhana
f3078d047c
Get bucketName from mux.Vars instead of heuristic code, resolves subdomain issues
2015-04-30 21:11:32 -07:00
Frederick F. Kautz IV
e911cc332e
Merge pull request #554 from fkautz/pr_out_disable_eviction_on_0_expiration_time
2015-04-30 19:40:46 -07:00
Harshavardhana
8e3d48bf35
Use UTC() everywhere
...
ref - https://github.com/golang/go/issues/1988 , no standard function to set timeZone
2015-04-30 19:38:32 -07:00
Frederick F. Kautz IV
94e24f1a05
Disable eviction on 0 expiration time
2015-04-30 19:34:35 -07:00
Harshavardhana
72395798c8
Merge pull request #552 from harshavardhana/pr_out_minio_config_directory_should_be_0700_not_os_modedir
2015-04-30 17:40:15 -07:00
Harshavardhana
2de81c8a3c
Minio config directory should be 0700 not os.ModeDir
2015-04-30 15:42:49 -07:00
Frederick F. Kautz IV
8e2f7f9493
Wrapping functions that require gc so that references are released before running
2015-04-30 12:38:12 -07:00
Harshavardhana
d815e6adfd
Changes to CreateObject() now returns back md5 along with any error
...
- This change is necessary to avoid the racy calls to GetObjectMetadata()
- This change is also necessary since one has to reply back md5sum with
PUT object response header
2015-04-30 03:38:14 -07:00
Harshavardhana
fd2203b1b7
Some more improvements to connection limit
2015-04-30 01:23:44 -07:00
Harshavardhana
f8a16dd22b
Memory now evicts bucket if no more objects in memory struct
...
- To avoid race in expiration while accessing memory driver structs with
in two competing write locks. Use lru Len() instead to know exact length
to schedule for eviction.
- squash both bucket and object structs, instead use a separate map to keep
mutable info of lastAccessTime which can be independently used inside the
expiration routine.
2015-04-30 00:31:52 -07:00
Frederick F. Kautz IV
ed0a9a9e06
Removing entity too large errors from memory driver
2015-04-30 00:09:54 -07:00
Frederick F. Kautz IV
aff2b3b06a
Adding debug statements for testing
2015-04-29 21:13:32 -07:00
Frederick F. Kautz IV
82a16b8848
Make minio work with curl and browsers again
2015-04-29 20:17:26 -07:00
Frederick F. Kautz IV
3a1386165f
Fixing race condition
2015-04-29 18:54:59 -07:00
Harshavardhana
03bca9a001
Fix out of memory issue
2015-04-29 18:33:19 -07:00
Harshavardhana
25d4c0f6fa
A small logical change in messaging and logging
2015-04-29 17:30:54 -07:00
Frederick F. Kautz IV
f596c744ed
Adding error message on connection limit met
2015-04-29 17:21:23 -07:00
Harshavardhana
a89c4a57d3
Return resources as its available, not a new value
2015-04-29 16:47:14 -07:00
Harshavardhana
251c99176a
Merge pull request #537 from harshavardhana/pr_out_add_proper_content_length_for_error_and_success_responses
2015-04-29 15:32:12 -07:00
Frederick F. Kautz IV
0da04c6f17
Expirations are now based on last accessed instead of time created
2015-04-29 15:28:45 -07:00
Harshavardhana
92e4301414
Add proper content-length for error and success responses
...
- All compliance issues with S3 API for Put,Get,List (Bucket,Object) respectively
- Encodes and returns back proper HTTP headers
2015-04-29 15:28:28 -07:00
Frederick F. Kautz IV
5fc5f79ab7
Adding lru
2015-04-29 15:08:26 -07:00
Frederick F. Kautz IV
8fcb697844
Simplifying memory sleep logic
2015-04-29 13:53:09 -07:00
Frederick F. Kautz IV
5cc9418ca7
Adding expiration
2015-04-29 13:02:21 -07:00
Frederick F. Kautz IV
a608524ebf
api logging is now in its own package
2015-04-29 12:24:53 -07:00
Harshavardhana
d1d5f7a78d
Collate success response into writeSuccessResponse(), add docs
2015-04-29 10:52:04 -07:00
Harshavardhana
b18bb230da
maxObjectSize and minObjectSize limitation added at putObjectHandler()
...
Put() replies back with
- EntityTooLarge with > 5GB in single PUT operation
- EntityTooSmall with < 1B in single PUT operation
- IncompleteBody with ho Content-Length found in HTTP request header
2015-04-29 02:19:53 -07:00
Frederick F. Kautz IV
172cd18455
Adding initial logging framework to api_router
2015-04-28 20:37:08 -07:00
Frederick F. Kautz IV
88cca647ea
Adding json logging to stderr
2015-04-28 18:11:53 -07:00
Frederick F. Kautz IV
de091068e8
Removing quotas, will handle with iptables
2015-04-28 12:14:34 -07:00
Frederick F. Kautz IV
f2257a643c
Critical sections are now protected by lock
2015-04-27 19:59:58 -07:00
Harshavardhana
605edbc307
Add some logging with debug
2015-04-27 19:26:24 -07:00
Harshavardhana
7bdcf4bbf3
Merge pull request #515 from fkautz/pr_out_fix_multiple_response_writeheader_calls
...
Fix multiple response.WriteHeader calls
2015-04-27 19:21:47 -07:00
Frederick F. Kautz IV
db5870be28
Fix multiple response.WriteHeader calls
2015-04-27 18:30:21 -07:00
Frederick F. Kautz IV
fc2537eef9
Merge pull request #513 from fkautz/pr_out_adding_software_fallback
2015-04-27 14:41:41 -07:00
Frederick F. Kautz IV
289c859675
Adding software fallback
2015-04-27 13:55:40 -07:00
Harshavardhana
ade803a923
Verify if request date is 5minutes late, reject such a request as
...
it could be a possible replay attack.
This commit also fixes #505 , by returning MethodNotAllowed instead of NotImplemented
2015-04-27 03:55:10 -07:00
Harshavardhana
9232ce3b4e
Now client requests for ACL changes are honored through PutBucketACL API
2015-04-27 03:04:29 -07:00
Harshavardhana
c99d96dbc2
Now donut supports bucket ACL's, bringing in this support for dl.minio.io
2015-04-27 02:09:32 -07:00
Harshavardhana
8beca83090
Limit memory mode at 100 buckets and return error appropriately
...
~~~
mc: <ERROR> Failed to create bucket for URL [http://localhost:9000/newbucket-101 ]. Reason: [You have attempted to create more buckets than allowed.].
~~~
Response from server
2015-04-26 22:15:39 -07:00
Harshavardhana
2d96d5ad57
Enhance error responses for request limit and bring some code from api errors
...
~~~
mc: <ERROR> Failed to create bucket for URL [http://localhost:9000/newbucket-101 ]. Reason: [Reduce your request rate.].
~~~
Client sees proper errors now.
2015-04-26 22:10:59 -07:00
Frederick F. Kautz IV
8a57006b36
Merge pull request #504 from fkautz/pr_out_making_lock_much_less_aggressive_fixes_multiple_uploads_blocking
2015-04-26 20:11:50 -07:00
Frederick F. Kautz IV
a974e4d7c7
Making lock much less aggressive, fixes multiple uploads blocking
2015-04-26 20:08:37 -07:00
Frederick F. Kautz IV
258bf55e31
Bandwidth quota now supports 100-Continue
2015-04-26 20:07:05 -07:00
Frederick F. Kautz IV
9431a54a27
finish me
2015-04-26 19:25:48 -07:00
Frederick F. Kautz IV
202fc26719
Revert "Exporting WriteErrorResponse"
...
This reverts commit e05d46a0b5
.
2015-04-26 17:02:49 -07:00
Frederick F. Kautz IV
e05d46a0b5
Exporting WriteErrorResponse
2015-04-26 16:28:54 -07:00
Frederick F. Kautz IV
64f7d04cd1
Adding connection limits
2015-04-26 16:14:51 -07:00
Frederick F. Kautz IV
7867ee89fa
Bandwidth quotas now work on data returned from server
2015-04-26 15:49:55 -07:00
Frederick F. Kautz IV
e2403cbc56
Quotas are more accurate, occur on Read()
2015-04-26 15:07:57 -07:00
Frederick F. Kautz IV
bbb493080f
Switching back to BigEndian
2015-04-25 18:16:21 -07:00
Frederick F. Kautz IV
3a48f9fe4d
Adding quotas based upon type
2015-04-25 18:07:02 -07:00
Harshavardhana
b75f46aaa0
Use standard functions to achieve byteorder converstion
2015-04-25 13:18:15 -07:00
Frederick F. Kautz IV
e2475925ea
quota handlers now log against ip properly against a duration
2015-04-24 20:49:33 -07:00
Harshavardhana
b3a9d8b058
Merge pull request #483 from harshavardhana/pr_out_memory_now_handles_submitting_large_files_fixes_482
2015-04-24 15:25:38 -07:00
Harshavardhana
c5d8ca245d
Memory now handles submitting large files - fixes #482
2015-04-24 15:22:22 -07:00
Frederick F. Kautz IV
fda50d52c8
Add initial framework for quota
2015-04-24 00:16:29 -07:00
Frederick F. Kautz IV
5a607a6762
Moving factories and start minio method to server package
2015-04-23 16:13:50 -07:00
Frederick F. Kautz IV
5c515d3138
Simplifying server config handling
2015-04-23 12:44:22 -07:00
Harshavardhana
5ae8d34f76
Donut mode will now take multiple paths as argument, updated docs as well
2015-04-23 11:36:14 -07:00
Harshavardhana
a5729e8c02
Disable verifying ACL until we get webcli up and running
2015-04-23 03:21:08 -07:00
Harshavardhana
bcc4449de7
Add proper help and several other cleanup
2015-04-23 02:42:50 -07:00
Harshavardhana
e95604ff86
Finishing all the test case support for ACL and other fixes
2015-04-23 01:24:46 -07:00
Frederick F. Kautz IV
0f8626ab1a
Refactoring cli to take commands
2015-04-23 01:05:44 -07:00
Harshavardhana
848c4ee31c
Further fixes for ACL support, currently code is disabled in all the handlers
...
Disabled because due to lack of testing support. Once we get that in we can
uncomment them back.
2015-04-22 19:31:45 -07:00
Harshavardhana
1c0ff2c758
ACL driver/storage layer support
2015-04-22 19:31:45 -07:00
Harshavardhana
c8713fd650
Bring in full fledged acl support
2015-04-22 18:26:35 -07:00
Harshavardhana
2c1455af1b
Implement x-amz-acl tests
2015-04-22 18:26:35 -07:00
Harshavardhana
107e077ec0
Implement x-amz-acl handling
2015-04-22 18:26:35 -07:00
Frederick F. Kautz IV
c56f7380d6
Memory driver now limits by size
2015-04-22 17:50:26 -07:00
Harshavardhana
b121c8588f
Remove bucketpolicy handlers and all its references
2015-04-22 14:46:51 -07:00
Harshavardhana
3f33643d39
File driver to take the high road
2015-04-16 02:16:39 -07:00
Frederick F. Kautz IV
59a81a73c6
Adding lru to memory driver, not wired to command line opts
2015-04-15 12:49:41 -07:00
Frederick F. Kautz IV
0cab3fd151
Better locking in memory driver
2015-04-15 11:48:56 -07:00
Harshavardhana
148935392d
Fix golint complaint
2015-04-08 23:42:59 -07:00
Frederick F. Kautz IV
26d8deee40
Removing unneeded files
2015-04-08 20:58:42 -07:00
Frederick F. Kautz IV
7212b3406a
Adding comments to iodine
2015-04-08 19:44:44 -07:00
Frederick F. Kautz IV
c12d5ea8a9
Adding iodine error conversion
2015-04-08 19:32:37 -07:00
Harshavardhana
38155e86ac
iodine is part of minio now
2015-04-08 18:05:48 -07:00
Harshavardhana
7fd1cc073c
Add everything back into one project hood, still missing iodine
2015-04-08 17:47:56 -07:00
Frederick F. Kautz IV
6241c9880b
Moving object storage drivers back to minio
2015-04-08 16:28:14 -07:00
Harshavardhana
2a21b7d639
Implement headBucketHandler() and its related tests
...
Remove internal testify/mock and use upstream - update all godeps
2015-04-07 01:16:32 -07:00
Frederick F. Kautz IV
ea961ef16d
Updating to use objectdriver
2015-04-06 18:00:18 -07:00