Harshavardhana
64f37bbf5b
rpc: Add RPC client tests. ( #2858 )
2016-10-06 02:30:54 -07:00
Karthic Rao
0fc96fa25c
Refactor bucket policy handler test to use API test initializer. ( #2859 )
2016-10-06 02:02:42 -07:00
Karthic Rao
2d8c6f8288
unit test for bucketPolicyConditionMatch function. ( #2857 )
2016-10-06 00:23:46 -07:00
Harshavardhana
b94211bd66
api: ListObjectsV1 compliance with AWS S3. ( #2856 )
...
XSD - xml schema definition for SOAP operations
on S3 provides positional restrictions on XML
output.
Fix the response by re-arranging the positions in
accordance with S3 behavior.
Fixes #2849
2016-10-05 20:12:47 -07:00
Harshavardhana
6494b77d41
server: Add more elaborate startup messages. ( #2731 )
...
These messages based on our prep stage during XL
and prints more informative message regarding
drive information.
This change also does a much needed refactoring.
2016-10-05 12:48:07 -07:00
Bala FA
63a7ca1af0
web: fix jwt token expiry set to one day by default. ( #2819 )
...
Fixes #2818
2016-10-05 10:18:55 -07:00
Krishna Srinivas
95f544657a
Signature-V2: use raw resource/query from the request for signature calculation. ( #2833 )
2016-10-05 09:18:53 -07:00
Harshavardhana
740a919e25
config: Use migrateV8 to v9 function properly. ( #2852 )
2016-10-05 02:28:04 -07:00
Krishnan Parthasarathi
402c92beda
Add listObjectParts test w/ unknown request signature type ( #2847 )
2016-10-04 07:57:35 -07:00
Karthic Rao
6a9013b97c
misspell fixes. ( #2835 )
2016-10-04 00:09:21 -07:00
Krishnan Parthasarathi
73b50aea2d
Add preSign auth type tests for ListObjectPartsHandler and PutObjectPartHandler ( #2834 )
2016-10-03 22:05:33 -07:00
Aditya Manthramurthy
315e66858c
Add PostgreSQL notifier ( #2739 ) ( #2824 )
...
* The user is required to specify a table name and database connection
information in the configuration file.
* INSERTs and DELETEs are done via prepared statements for speed.
* Assumes a table structure, and requires PostgreSQL 9.5 or above due to
the use of UPSERT.
* Creates the table if it does not exist with the given table name using
a query like:
CREATE TABLE myminio (
key varchar PRIMARY KEY,
value JSONB
);
* Vendors some required libraries.
2016-10-03 17:29:55 -07:00
Krishnan Parthasarathi
4f902d42b2
Add unit-tests for ListObjectParts API handler ( #2826 )
...
* Add missing uploadID test
... make variables in test code unexported.
* Add ServerNotInitialized test for ListObjectPartsHandler
* Add tests for ListObjectParts with signatureV2 and Anonymous requests
* Add failure test cases for ListObjectParts
2016-10-03 08:54:57 -07:00
Krishna Srinivas
61a18ed48f
sha256: Verify sha256 along with md5sum, signature is verified on the request early. ( #2813 )
2016-10-02 15:51:49 -07:00
Anis Elleuch
b5a6dd1395
Avoid path-cleaning policy resources for a better compliance with S3 ( #2823 )
2016-10-01 21:30:25 -07:00
Krishnan Parthasarathi
83e6e1060e
Layer LimitReader responsibly allowing sign verification to work ( #2821 )
2016-10-01 09:37:40 -07:00
Krishnan Parthasarathi
ddeb8242d8
PutObjectPartHandler unit-tests ( #2810 )
2016-10-01 08:23:26 -07:00
Harshavardhana
5ecba587f7
api: Relax object name validation. ( #2814 )
...
Fixes #2812
2016-09-30 16:56:36 -07:00
Harshavardhana
db3da97a50
signature/v2: Fix presigned requests.
2016-09-30 15:22:00 -07:00
Harshavardhana
5885ffc8ae
signature: Add legacy signature v2 support transparently. ( #2811 )
...
Add new tests as well.
2016-09-30 14:32:13 -07:00
Anis Elleuch
9fb1c89f81
Add TLS encryption capability to RPC clients ( #2789 )
2016-09-29 23:42:37 -07:00
Anis Elleuch
1e6afac3bd
Add NATS notifier ( #2795 )
2016-09-29 23:42:10 -07:00
Harshavardhana
64083b9227
signature: Region changes should be handled just like AWS. ( #2805 )
...
- PutBucket happens with 'us-east-1'.
- ListBuckets happens with any region.
- GetBucketLocation happens with 'us-east-1' and location is returned.
2016-09-29 15:51:00 -07:00
Krishnan Parthasarathi
5fdd768903
Make addition of TopicConfig to globalEventNotifier go-routine safe ( #2806 )
2016-09-28 22:46:19 -07:00
Krishnan Parthasarathi
428629f577
Add unit tests for server-main.go ( #2802 )
2016-09-28 11:19:07 -07:00
Harshavardhana
1edd74dda2
update: Deprecate the usage of update=yes query param. ( #2801 )
...
Fixes #2799
2016-09-28 02:41:21 -07:00
Krishnan Parthasarathi
740ecf530c
Add PutBucketNotification, ListenBucketNotification handler unit-tests. ( #2787 )
2016-09-28 01:08:03 -07:00
Aditya Manthramurthy
10d2ef5449
Remove comments relating to deprecated MINIO_DEBUG envvar ( #2797 )
2016-09-27 18:28:46 -07:00
Aditya Manthramurthy
8ea571c7f7
Remove MINIO_DEBUG environment variable ( #2794 )
...
Removes the unimplemented settings of MINIO_DEBUG=mem and makes
MINIO_DEBUG=lock the default behaviour.
2016-09-27 14:35:43 -07:00
Harshavardhana
ca3022d545
api: Change ListenBucketNotification with new API format. ( #2791 )
...
Take prefix, suffix and events as query params.
2016-09-27 13:17:43 -07:00
Anis Elleuch
9417614a8e
Recalculate free minimum disk space ( #2788 )
...
* Fix calculating free space disk by using blocks available for unprivileged user
* Use fixed minimal free disk space instead of percentage
2016-09-27 12:46:38 -07:00
Aditya Manthramurthy
70d52bbc4c
Add unit test for rate-limit-handler ( #2661 ) ( #2784 )
2016-09-26 21:31:12 -07:00
Harshavardhana
6aa2fc95c0
Revert "bucket: refactor policies and fix bugs related to enforcing policies. ( #2766 )"
...
This reverts commit ca5ca8332b
.
2016-09-26 19:32:33 -07:00
Harshavardhana
cfbab22237
web: Remove bucket policy when we have no more statements. ( #2779 )
2016-09-26 03:11:22 -07:00
Harshavardhana
be0e06c0aa
web: Simplify and rename GetAllBucketPolicy --> ListAllBucketPolicies. ( #2778 )
2016-09-25 21:53:19 -07:00
Harshavardhana
1c941fd787
rpc: Should validate server versions. ( #2775 )
...
Fixes #2764
2016-09-24 03:34:45 -07:00
Krishnan Parthasarathi
669783f875
Purge stale object cache entry ( #2770 )
2016-09-23 19:55:28 -07:00
Krishnan Parthasarathi
27e474b3d2
Improve code coverage in bucket-notification-handlers.go ( #2759 )
...
* Fix incorrect test cases for bucket-notification handler
* Add tests covering failure cases for bucket notification
2016-09-23 13:32:51 -07:00
Krishna Srinivas
1e53316241
Add tests for presigned-get ( #2767 )
...
* web-handlers: support for presigned-get json-rpc call for MinioBrowser's "share" feature.
* Add tests for presigned-get
2016-09-23 01:25:49 -07:00
Harshavardhana
ca5ca8332b
bucket: refactor policies and fix bugs related to enforcing policies. ( #2766 )
...
This patch also addresses the problem of double caching at
object layer once at XL and another at handler layer.
2016-09-22 23:47:48 -07:00
Bala FA
aa579bbc20
web: add method to get all policies for given bucket name. ( #2756 )
...
Refer #1858
2016-09-22 23:06:45 -07:00
Harshavardhana
e375d822da
bucket: SetBucketPolicy should save a valid Version and validate. ( #2762 )
2016-09-22 22:27:21 -07:00
Anis Elleuch
fc783f8407
More tests for web handlers ( #2755 )
...
* Return negative values of Total and Free in StorageInfo() when we fail to get disk info
* Return consistent messages in web handlers when the server is not initialized
2016-09-22 16:35:12 -07:00
Anis Elleuch
ef22330563
Require go 1.7.1 to build Minio server ( #2727 )
2016-09-22 10:33:52 -07:00
Karthic Rao
1148f95292
ineffassign fixes ( #2758 )
2016-09-21 23:03:54 -07:00
Karthic Rao
f7430ec09c
use runtime/debug.Stack() in leak detect test ( #2757 )
2016-09-21 22:04:35 -07:00
Karthic Rao
b8903d842c
api/complete-multipart: fixes and tests. ( #2719 )
...
* api/complete-multipart: tests and simplification.
- Removing the logic of sending white space characters.
- Fix for incorrect HTTP response status for certain cases.
- Tests for New Multipart Upload and Complete Multipart Upload.
* tests: test for Delelete Object API handler
2016-09-21 20:08:08 -07:00
Aditya Manthramurthy
32f097b4d6
Controller rpc tests ( #2709 )
...
* Test code for controller-handler operations:
* Heal operations
* List operation
* Switch to "testing" lib, moving away from gocheck
* Minor refactors
* Remove extra call to initGracefulShutdown
* Remove dead code in mainControl:
Dead code found by the TestControlMain() test function that always
passes.
* Add tests for control-*-main.go
2016-09-21 19:58:50 -07:00
Krishnan Parthasarathi
559ad38b8c
Add bucket-notification-handler tests ( #2750 )
2016-09-21 17:41:34 -07:00
Anis Elleuch
90417d2dd6
Check for bucket existence in Set/Get/Remove bucket policy workflow + tests ( #2745 )
2016-09-21 16:38:50 -07:00
Anis Elleuch
e66fb4bd7b
configMigrate() returns errors + tests ( #2735 )
2016-09-21 09:44:57 -07:00
Harshavardhana
018c90dae7
events: ElasticSearch doesnt support objects with '/' in them. ( #2747 )
...
Fix this by using a unique sha256 generated for each unique key.
2016-09-20 16:36:18 -07:00
Anis Elleuch
a5066e8f76
Better code coverage of handler-utils.go ( #2746 )
2016-09-20 10:40:46 -07:00
Harshavardhana
0a3448c8b6
events: Change event notifiers to delete and update keys. ( #2742 )
...
ElasticSearch and Redis are both treated like a database.
Each indexs are based on the object names uniquely indentifying
the event. Upon each delete event of the named object deletes
the index on elasticsearch and redis respectively.
2016-09-20 02:11:17 -07:00
Harshavardhana
c4964232eb
config: Fail to start for config mistakes. ( #2740 )
2016-09-19 15:23:49 -07:00
Harshavardhana
ef3c807b4a
policies: Parser should handle Principals with various forms. ( #2733 )
...
Handles cases for these three combinations
- "Principal": "*",
- "Principal": { "AWS" : "*" }
- "Principal": { "AWS" : [ "*" ]}
Fixes #2732
2016-09-19 13:52:28 -07:00
Harshavardhana
113b93346b
lock: Make some cleanup and moving the code around. ( #2718 )
...
This patch just avoids lot of ifs and inverts some logic.
2016-09-19 13:14:55 -07:00
Krishna Srinivas
a955676986
Signature-V4: Dump the request with error message on signature mismatch. ( #2734 )
...
fixes #2691
2016-09-19 10:17:46 -07:00
Harshavardhana
725df557b5
tests: Add tests for bucket-notification-utils ( #2726 )
...
Part - 2 final fix #2711
2016-09-17 03:19:39 -07:00
Harshavardhana
490056eee3
tests: Add tests for bucket-notification-utils ( #2723 )
...
Part fix - 1 for #2711
2016-09-16 17:26:27 -07:00
Harshavardhana
797d749322
tests: Add tests for filterRuleMatch ( #2722 )
...
Part-1 fix for #2418
2016-09-16 16:44:44 -07:00
Harshavardhana
79888bfff7
tests: Add auth-handler. ( #2721 )
...
Fixes #2658
2016-09-16 15:17:49 -07:00
Anis Elleuch
010f61e91f
Add more tests for event-notifier code ( #2716 )
2016-09-16 14:26:05 -07:00
Harshavardhana
9216981262
tests: Add test for diskCount. ( #2717 )
...
Fixes #2312
2016-09-16 13:44:52 -07:00
Anis Elleuch
b89a1cd482
tests: Implemented more tests for fs-v1*.go ( #2686 )
2016-09-16 13:06:49 -07:00
Harshavardhana
7d37dea449
tests: Add more streaming signature tests. ( #2713 )
...
Part fix for #2621
2016-09-16 02:45:42 -07:00
Frank
df2ef64d20
Upgrade to new dsync version incl. stale lock detection ( #2708 )
2016-09-16 00:30:55 -07:00
Anis Elleuch
7a549096de
XL and FS use different tree walk ignored errors ( #2707 )
2016-09-15 13:43:40 -07:00
Harshavardhana
a1ff351f21
tests: Fix ListMultipartUploadsHandler tests. ( #2705 )
2016-09-15 01:44:19 -07:00
Harshavardhana
03430d0db8
tests: Add ListBucketHandler tests. ( #2701 )
...
part-3 final fix for #2412
2016-09-14 23:53:42 -07:00
Anis Elleuch
6f73d597e0
Fix tracing twice an error in fs Complete Multipart Upload ( #2703 )
2016-09-14 21:24:54 -07:00
Anis Elleuch
a84548d7ea
Fix FS remove bucket regression bug ( #2693 )
2016-09-14 16:41:39 -07:00
Harshavardhana
1e6d67b16d
server: Remove deadcode. ( #2699 )
2016-09-14 13:43:08 -07:00
Aditya Manthramurthy
6533927237
Lock-free rate-limit algorithm + bug-fix ( #2694 )
2016-09-14 11:27:37 -07:00
Harshavardhana
da9ae574df
server: We should fail properly during server startup. ( #2689 )
...
Fixes #2688
2016-09-14 01:11:03 -07:00
Harshavardhana
ee7e70c992
tests: Add tests for ListMultipartUploads, DeleteMultipleObjects. ( #2649 )
...
Additionally adds PostPolicyHandler tests.
2016-09-13 21:22:31 -07:00
Krishna Srinivas
54a9f59a13
Init: Print SQS ARNs after globalEventNotifier is inited. ( #2682 )
...
fixes #2681
2016-09-13 21:18:30 -07:00
Harshavardhana
e6fd664331
tests: Fix format-config tests.
2016-09-13 21:18:30 -07:00
Karthic Rao
b247ec9352
tests: refactor object-handler tests. ( #2656 )
...
- Move the initialization to a common executor for object Layer API
tests.d
2016-09-13 21:18:30 -07:00
Harshavardhana
43befab8ef
Change distributed server wording.
2016-09-13 21:18:30 -07:00
Harshavardhana
eae0281c64
tests: Add GetBucketLocation, HeadBucket tests. ( #2644 )
2016-09-13 21:18:30 -07:00
Karthic Rao
8bd78fbdfb
performance: gjson parsing for readXLMeta, listParts, getObjectInfo. ( #2631 )
...
- Using gjson for constructing xlMetaV1{} in realXLMeta.
- Test for parsing constructing xlMetaV1{} using gjson.
- Changes made since benchmarks showed 30-40% improvement in speed.
- Follow up comments in issue https://github.com/minio/minio/issues/2208
for more details.
- gjson parsing of parts from xl.json for listParts.
- gjson parsing of statInfo from xl.json for getObjectInfo.
- Vendorizing gjson dependency.
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
66459a4ce0
Add unit-tests for formatting disks during initialization ( #2635 )
...
* Add unit-tests for formatting disks during initialization
- Fixed corresponding code at places where it was deviating from the
tabular spec.
* Added more test cases and simplified algo
... based on feedback from ``go test -coverprofile``.
2016-09-13 21:18:30 -07:00
Harshavardhana
182109f0de
xl: Heal format.json
properly on fresh disks.
2016-09-13 21:18:30 -07:00
Harshavardhana
9998e9ea19
api: Response timeFormat do not need to have nano-second precision.
...
Fixes an error reported by s3verify.
2016-09-13 21:18:30 -07:00
Harshavardhana
ba2ba328da
server: Fixes for various conditions
...
- Fix distributed branch to be able to run FS version.
- Fix distributed branch to be able to run XL local disks.
- Ignore initialization failures of notification and bucket
policies, the codepath should load whatever is possible.
2016-09-13 21:18:30 -07:00
Anis Elleuch
67b8080144
Fix control lock rpc name in control lock cmd ( #2627 )
2016-09-13 21:18:30 -07:00
Anis Elleuch
239a34ca97
Add tests for regular and streaming v4 PutObject Handler ( #2618 )
2016-09-13 21:18:30 -07:00
Krishna Srinivas
81d8263ae2
binary-update: Do not fetch update info for minio binary compiled from source.
...
fixes #2494
2016-09-13 21:18:30 -07:00
Krishna Srinivas
b4e4846e9f
PutObject: object layer now returns ObjectInfo instead of md5sum to avoid extra GetObjectInfo call. ( #2599 )
...
From the S3 layer after PutObject we were calling GetObjectInfo for bucket notification. This can
be avoided if PutObjectInfo returns ObjectInfo.
fixes #2567
2016-09-13 21:18:30 -07:00
Krishna Srinivas
92e49eab5a
FS/Multipart: Do not rename append files to another tmp file as the append files are already in tmp location. ( #2612 )
2016-09-13 21:18:30 -07:00
Harshavardhana
c4a7b950a0
fs: Fix asynchronous multipart bug.
...
Construct part path properly.
2016-09-13 21:18:30 -07:00
Karthic Rao
1ce339abeb
Fixing ineffssaign errors ( #2608 )
2016-09-13 21:18:30 -07:00
Anis Elleuch
3e284162d7
Add global flags to all commands and subcommands ( #2605 )
2016-09-13 21:18:30 -07:00
Anis Elleuch
ff99392102
Enhance minio server help template ( #2603 )
2016-09-13 21:18:30 -07:00
Krishna Srinivas
9358ee011b
logging: Print stack trace in case of errors.
...
fixes #1827
2016-09-13 21:18:30 -07:00
Harshavardhana
37cbcae6ba
xl: Remove an unecessary lock with isBucketExist() ( #2593 )
...
Fixes #2566
2016-09-13 21:18:30 -07:00
Harshavardhana
ae64b7fac8
XL: Handle object layer initialization properly.
...
Initialization when disk was down the network disk
reported an incorrect error rather than errDiskNotFound.
This resulted in incorrect error handling during
prepInitStorage() stage.
Fixes #2577
2016-09-13 21:18:30 -07:00
Anis Elleuch
d936ed90ae
Avoid testing on system errors strings in posix ( #2583 )
2016-09-13 21:18:30 -07:00
Krishna Srinivas
7cc77eba45
XL/Healing: errDiskNotFound is the only pardonable error in xlShouldHeal. ( #2586 )
...
This is so that we try to heal a file for all the "bad" cases except when the disk is down.
2016-09-13 21:18:30 -07:00