Commit graph

73 commits

Author SHA1 Message Date
Harshavardhana 5aa5dcdc6d
lock: improve locker initialization at init (#8776)
Use reference format to initialize lockers
during startup, also handle `nil` for NetLocker
in dsync and remove *errorLocker* implementation

Add further tuning parameters such as

 - DialTimeout is now 15 seconds from 30 seconds
 - KeepAliveTimeout is not 20 seconds, 5 seconds
   more than default 15 seconds
 - ResponseHeaderTimeout to 10 seconds
 - ExpectContinueTimeout is reduced to 3 seconds
 - DualStack is enabled by default remove setting
   it to `true`
 - Reduce IdleConnTimeout to 30 seconds from
   1 minute to avoid idleConn build up

Fixes #8773
2020-01-10 02:35:06 -08:00
Harshavardhana c8d82588c2 Fix crash in console logger and also handle bucket DNS updates (#8654)
Also fix listenBucketNotification bugs seen by minio-js
listen bucket notification API.
2019-12-16 20:30:57 -08:00
Harshavardhana 3e9ab5f4a9
Fix k8s replica set deployment (#8629)
In replica sets, hosts resolve to localhost
IP automatically until the deployment fully
comes up. To avoid this issue we need to
wait for such resolution.
2019-12-10 20:28:22 -08:00
Harshavardhana e9b2bf00ad Support MinIO to be deployed on more than 32 nodes (#8492)
This PR implements locking from a global entity into
a more localized set level entity, allowing for locks
to be held only on the resources which are writing
to a collection of disks rather than a global level.

In this process this PR also removes the top-level
limit of 32 nodes to an unlimited number of nodes. This
is a precursor change before bring in bucket expansion.
2019-11-13 12:17:45 -08:00
Harshavardhana ee4a6a823d Migrate config to KV data format (#8392)
- adding oauth support to MinIO browser (#8400) by @kanagaraj
- supports multi-line get/set/del for all config fields
- add support for comments, allow toggle
- add extensive validation of config before saving
- support MinIO browser to support proper claims, using STS tokens
- env support for all config parameters, legacy envs are also
  supported with all documentation now pointing to latest ENVs
- preserve accessKey/secretKey from FS mode setups
- add history support implements three APIs
  - ClearHistory
  - RestoreHistory
  - ListHistory
- add help command support for each config parameters
- all the bug fixes after migration to KV, and other bug
  fixes encountered during testing.
2019-10-22 22:59:13 -07:00
Harshavardhana 6a4ef2e48e Initialize configs correctly, move notification config (#8367)
This PR also removes deprecated tests, adds checks
to avoid races reproduced on CI/CD.
2019-10-09 11:41:15 +05:30
Harshavardhana 290ad0996f Move etcd, logger, crypto into their own packages (#8366)
- Deprecates _MINIO_PROFILER, `mc admin profile` does the job
- Move ENVs to common location in cmd/config/
2019-10-08 11:17:56 +05:30
Harshavardhana cb01516a26 In HDFS gateway fix non-empty folder behavior (#8254)
To be compatible with our FS and Erasure coded
mode deployments, make sure that we do not send
200 OK for folders which have files inside.

Fixes #8143
2019-09-18 01:59:59 +05:30
Harshavardhana 94e5cb7576
Migrate to go1.13 to avail all new features (#8203)
Read more https://blog.golang.org/go1.13
2019-09-08 16:44:15 -07:00
Harshavardhana add1b6cb6b
Remove go mod cache due to buggy travis caching (#8113)
There are sporadic failures due to go-mod build
caches on travis, deprecate it since we moved to
using GOPROXY.
2019-08-21 13:38:52 -10:00
Harshavardhana b83413b167 Use GOPROXY to speed up builds (#7984)
Read more here https://proxy.golang.org proposal 
for go1.13
2019-07-30 22:27:11 +05:30
Harshavardhana 1dc25bcf5f
Add mint tests into MinIO repo (#7886) 2019-07-09 18:32:39 -07:00
Harshavardhana 720ed3f5e8 Add go-mod cache for .travis to speed up builds (#7477) 2019-04-05 12:00:31 +05:30
Harshavardhana 313a3a286a Migrate to go1.12 to simplify our cmd/http package (#7302)
Simplify the cmd/http package overall by removing
custom plain text v/s tls connection detection, by
migrating to go1.12 and choose minimum version
to be go1.12

Also remove all the vendored deps, since they
are not useful anymore.
2019-04-02 18:28:39 -07:00
Sidhartha Mani 6bc0de2a75 add go modules file and start running go 1.11 style builds (#7354) 2019-03-19 13:50:58 -07:00
Aditya Manthramurthy f97a33a63f Downgrade node temporarily (#7333)
To fix jest related build failure as recommended at https://github.com/facebook/jest/issues/8069#issuecomment-470307590
2019-03-06 16:05:49 -08:00
Harshavardhana 8e0910ab3e Fix build issues on BSDs in pkg/cpu (#7116)
Also add a cross compile script to test always cross
compilation for some well known platforms and architectures
, we support out of box compilation of these platforms even
if we don't make an official release build.

This script is to avoid regressions in this area when we
add platform dependent code.
2019-01-22 09:27:23 +05:30
Bala FA b0deea27df Refactor s3select to support parquet. (#7023)
Also handle pretty formatted JSON documents.
2019-01-08 16:53:04 -08:00
Harshavardhana b5280ba243
Migrate to Go version 1.11.4 (#7026) 2018-12-28 14:04:39 -08:00
Harshavardhana bebaff269c Support IPv6 in minio command line (#6947)
Fixes #6946
2018-12-14 13:07:46 +05:30
Harshavardhana 30040fba45 Add windows CI on travis (#6661) 2018-10-18 08:48:53 +05:30
Harshavardhana a13cd7b7c4 Separate build functional tests from Makefile (#6351)
Recently travis seems to have issues with builds after
merge, so this PR is an experiment to fix this.

Failed: https://travis-ci.org/minio/minio/builds/419769285#L2310

Succeeded: https://travis-ci.org/minio/minio/builds/419565606#L2322

This PR was tested to be working properly on my fork to build
on travis, would need to do the same in minio/minio and see
how it goes through.
2018-08-28 13:57:01 +05:30
Harshavardhana 7d7e21aebb Merge initConfig logic to ConfigSys (#6312) 2018-08-19 13:57:18 -07:00
kannappanr 7b91bd71fe Remove ppc64le support from .travis.yml (#6180)
Travis CI for ppc64le stays in queued state.
Removing the support for now.
2018-07-20 18:39:55 -07:00
Rafael Peria de Sene 317e648c0d Add support for ppc64le (#6116)
Signed-off-by: Rafael Peria de Sene <rpsene@br.ibm.com>
2018-07-03 09:42:19 -07:00
Harshavardhana 73f7a98590 Enable go1.10.1 version (#5666) 2018-04-06 11:07:39 -07:00
Kanagaraj M 19451e374a revert browser newux changes (#5714) 2018-03-26 12:49:12 -07:00
Kanagaraj M 1c91c7b54d added eslint prettier check to travis (#5701)
- "yarn eslint" will throw the error if there are any formatting issues
- "yarn format" will format the js file based on the project's
prettier configuration.
2018-03-23 11:41:10 -07:00
Kanagaraj M 4ca0ec4586 added yarn test to travis.yml (#5696) 2018-03-23 14:36:20 +05:30
Harshavardhana 6b3db7556a Fix gofmt issues reported for simplification (#5581)
added travis to catch this issue
2018-02-26 23:52:03 +05:30
Harshavardhana fd3897d0c3 Move to go1.9.4 with recent security release (#5502) 2018-02-08 14:33:22 +05:30
Bala FA d28b3d8801 Move to go1.9.1 as default environment. (#5041) 2017-10-09 22:23:59 -07:00
Bala FA d8a11c8f4b fix build failure for go1.9 (#4872) 2017-10-06 17:00:15 -07:00
Davor Kapsa aceaa1ec48 travis: update go versions (#4945) 2017-09-22 14:05:07 -07:00
Harshavardhana 58633a383a Deprecate intel-32, arm64, arm support for minio builds. (#4811)
It was decided that we will be deprecating ARM support
for minio builds. ARM users should simply compile from source.

Additionally 32bit version of Linux, Windows and FreeBSD (64bit)
are deprecated.
2017-08-14 11:08:58 -07:00
A. Elleuch b4dc6df35c go1.8: Changes to support golang 1.8 (#4759)
QuirkConn is added to replace net.Conn as a workaround to a golang bug:
https://github.com/golang/go/issues/21133
2017-08-06 11:27:33 -07:00
Harshavardhana 432bf7d99e Fail if formatting is wrong in our CI tests. (#4459)
We didn't fail before, we should helps in avoiding
formatting issues to creep into the codebase.
2017-06-02 14:05:51 -07:00
Harshavardhana e3b627a192 docker: Add ARM64 image build support (#3876) 2017-03-09 13:57:27 -08:00
Harshavardhana 433225ab0d docker: ca-certificates should not be removed. (#3868) 2017-03-07 16:30:19 -08:00
Harshavardhana 3e655a2c85 docker: Add ARM docker container dockerfile. (#3574) 2017-03-07 15:15:05 -08:00
Harshavardhana 99a12613a3 update: For source builds look for absolute path. (#3780)
os.Args[0] doesn't point to absolute path we need
use exec.LookPath to find the absolute path before
sending os.Stat().
2017-02-21 01:32:05 -08:00
Harshavardhana 31dff87903 Honor envs properly for access and secret key. (#3703)
Also changes the behavior of `secretKeyHash` which is
not necessary to be sent over the network, each node
has its own secretKeyHash to validate.

Fixes #3696
Partial(fix) #3700 (More changes needed with some code cleanup)
2017-02-07 12:51:43 -08:00
Harshavardhana ac9ba13c19 build: Increase the travis build test timeout to 15mins 2017-01-30 01:39:53 -08:00
Harshavardhana ccd949d8ca Add -v for race tests to avoid travis and appveyor timeouts. (#3647)
This is added so that CI's don't wrong fully timeout on
little longer running tests.
2017-01-27 12:31:02 -08:00
Alex d6a327fbc5 Add notifications by webhook.
Add a new config entry moving to version 13.
```
		"webhook": {
			"1": {
				"enable": true,
				"address": "http://requestb.in/1i9al7m1"
			}
		}
```
2017-01-12 10:19:59 -08:00
Harshavardhana d31f256020 Fail on lint errors during CI build. 2016-12-02 18:08:12 -08:00
Harshavardhana e9c45102b0 posix: Use sync.Pool buffers to copy in large buffers. (#3106)
These fixes are borrowed from the fixes required for GlusterFS i/o throughput.
2016-10-26 17:14:05 -07:00
Harshavardhana 83b364891d tests: Fix a potential race in ListenBucketNotificationHandler. (#3040) 2016-10-21 22:56:27 -07:00
Harshavardhana ecaccefd2e tests: Implement GetBucketNotification handler tests. (#3029) 2016-10-21 02:39:37 -07:00
Mike Ralphson 7fc1685b7a Allow Travis builds from GitHub forks (#2958)
Set the go_import_path explicitly. See
https://docs.travis-ci.com/user/languages/go#Go-Import-Path
2016-10-17 08:42:20 -07:00