Commit graph

272 commits

Author SHA1 Message Date
Harshavardhana e0abb46616 fix: go mod tidy 2021-08-23 17:32:29 -07:00
Anis Elleuch 170b89f468
Update minio-go library (#13045)
This fixes some issues in replication & S3 gateway:
- https://github.com/minio/minio-go/pull/1531
- https://github.com/minio/minio-go/pull/1533
2021-08-23 17:31:53 -07:00
Anis Elleuch 7fb9301c03
heal: Return parity for storage classes in heal info API (#13038)
`mc admin heal` command will show servers/disks tolerance, for that
purpose, you need to know the number of parity disks for each storage
class.

Parity is always the same in all pools.
2021-08-23 08:50:35 -07:00
Harshavardhana 6167104644 update console to v0.9.1 2021-08-17 13:53:08 -07:00
Klaus Post 92bb2928e4
Compress better on amd64 (#12974)
Since S2 has amd64 assembly, it now operates at a reasonable 
speed to use by default.

Here are some examples of stream compression speed, 16 cores:
```
nyc-taxi-data-10M.csv	s2	1	3325605752	-> 1095998837	312ms	10139.07MB/s		67.04% reduction
nyc-taxi-data-10M.csv	s2	2	3325605752	-> 917905514	428ms	7393.74MB/s		72.40%

github-june-2days-2019.json	s2	1	6273951764	-> 1043196283	391ms	15301.99 MB/s		83.37%
github-june-2days-2019.json	s2	2	6273951764	-> 955924506	519ms	11510.81MB/s		84.76%

github-ranks-backup.bin	s2	1	1862623243	-> 623911363	146ms	12133MB/s		66.50%
github-ranks-backup.bin	s2	2	1862623243	-> 563752759	230ms	7705.26MB/s		69.73%
```

We keep non-assembly platforms on the faster, but less efficient mode.
2021-08-16 11:55:07 -07:00
Shireesh Anjal d44e4399e6
Add admin api to return sys services info (#12939)
The intention is to provide status of any sys services that can
potentially impact the performance of minio.

At present, it will return information about the `selinux` service
(not-installed/disabled/permissive/enforcing)

Signed-off-by: Shireesh Anjal <shireesh@minio.io>
2021-08-12 18:58:40 -07:00
Harshavardhana 77b8885a24 update console to latest release v0.9.0 2021-08-12 18:08:30 -07:00
Klaus Post 9ab5e0312d
Simplify gzhttp wrapper (#12912)
The wrapper now accepts interfaces so we don't need a wrapper for that any more.
2021-08-09 12:45:59 -07:00
Harshavardhana 6c0757eea6
fix: simplify APIEndpoints() usage (#12893)
improvements include

- skip IPv6 correctly
- do not set default value for
  MINIO_SERVER_URL, let it be
  configured if not use local IPs

Bonus:

- In healing return error from listPathRaw()
- update console to v0.8.3
2021-08-05 15:01:19 -07:00
Harshavardhana 510c67de4a update console to v0.8.2 2021-08-02 13:07:49 -07:00
Harshavardhana 4567160e4d update madmin-go to v1.0.20 2021-07-29 17:02:00 -07:00
Harshavardhana bc6a2b579b update to console v0.8.1 2021-07-29 16:14:10 -07:00
Harshavardhana 3735450e7e
fix: allow audience claim to be an array (#12810)
Some incorrect setups might have multiple audiences
where they are trying to use a single authentication
endpoint for multiple services.

Nevertheless OpenID spec allows it to make it
even more confusin for no good reason.

> It MUST contain the OAuth 2.0 client_id of the
> Relying Party as an audience value. It MAY also
> contain identifiers for other audiences. In the
> general case, the aud value is an array of case
> sensitive strings. In the common special case
> when there is one audience, the aud value MAY
> be a single case sensitive string.

fixes #12809
2021-07-27 18:37:51 -07:00
Krishna Srinivas aa0c28809b
Server side speedtest implementation (#12750) 2021-07-27 12:55:56 -07:00
Harshavardhana e936871b83 update to console v0.8.0 2021-07-23 19:53:56 -07:00
Harshavardhana 922f5da8dc update nats.io to fix critical bug 2021-07-23 18:33:31 -07:00
Harshavardhana 294bafd58b remove unused WebTrace and minio/rpc dependency 2021-07-23 15:50:21 -07:00
Harshavardhana b865b9f43d update console to fix assets 2021-07-21 22:23:32 -07:00
Harshavardhana 4a2cb23690 update console to master 2021-07-21 15:15:23 -07:00
Harshavardhana 7f45e80755 update console to master with more UI fixes 2021-07-21 11:19:36 -07:00
Harshavardhana ab7584eeac update to latest console fs/gateway 2021-07-19 17:49:54 -07:00
Harshavardhana 27e07e80bc
fix: accountInfo() error FS mode (#12734) 2021-07-17 01:17:35 -07:00
Harshavardhana b13bca85d0
send backendInfo as part AccountInfo (#12733)
This is needed for console UI to change
the UI behavior for different modes of
operation.
2021-07-16 14:37:06 -07:00
Anis Elleuch b0b4696a64
heal: Add MRF metrics to background heal API response (#12398)
This commit gathers MRF metrics from 
all nodes in a cluster and return it to the caller. This will show information about the 
number of objects in the MRF queues 
waiting to be healed.
2021-07-15 22:32:06 -07:00
Harshavardhana 9516587a6c update console to master branch
with new fixes

- improve download behavior
- avoid response timeouts
2021-07-15 14:44:18 -07:00
Anis Elleuch a4b8928660
Move prefix usage in admin AccountInfo API (#12710) 2021-07-14 08:51:10 -07:00
Shireesh Anjal 4d6d4244f1
Anonymize hosts in subnet health report (#12627)
Ensure that hostnames / ip addresses are not printed in the subnet
health report. Anonymize them by replacing them with `servern` where `n`
represents the position of the server in the pool.

This is done by building a `host anonymizer` map that maps every
possible value containing the host e.g. host, host:port,
http://host:port, etc to the corresponding anonymized name and using
this map to replace the values at the time of health report generation.

A different logic is used to anonymize host names in the `procinfo`
data, as the host names are part of an ellipses pattern in the process
start command. Here we just replace the prefix/suffix of the ellipses
pattern with their hashes.
2021-07-14 00:23:22 -07:00
Harshavardhana c438eda624 update console dependency for derivedKey fix 2021-07-12 14:37:25 -07:00
Klaus Post 3201bae773
feat: gzip S3/Admin API responses whenever applicable (#12631)
Gzip responses if appropriate, except GetObject requests.
List reponses has an almost 10:1 compression ratio with no
measurable slowdown (in fact it seems a bit faster).
2021-07-12 10:49:50 -07:00
Anis Elleuch 8625cfdfae
update madmin-go dependency, update related code (#12669) 2021-07-12 10:16:10 -07:00
Minio Trusted e4c3953947 update console dependency 2021-07-10 12:16:07 -07:00
Klaus Post d6a2fe02d3
Add admin file inspector (#12635)
Download files from *any* bucket/path as an encrypted zip file.

The key is included in the response but can be separated so zip 
and the key doesn't have to be sent on the same channel.

Requires https://github.com/minio/pkg/pull/6
2021-07-09 11:29:16 -07:00
Harshavardhana 6503c6ac21 fix: support startAfter with S3 gateway
fixes #12604
2021-07-05 18:44:04 -07:00
Harshavardhana bb92989359 fix: cross compilation on openbsd/amd64
fixes #12620
2021-07-04 00:34:10 -07:00
Krishnan Parthasarathi a1df230518
Add a 'free' version to track deletion of tiered object content (#12470) 2021-06-30 19:32:07 -07:00
Poorna Krishnamoorthy a3f0288262
Use multipart call for replication (#12535)
if object was uploaded with multipart. This is to ensure that
GetObject calls with partNumber in URI request parameters
have same behavior on source and replication target.
2021-06-30 07:44:24 -07:00
Harshavardhana 75ce633f0a update minio/console to latest master 2021-06-29 12:06:27 -07:00
Harshavardhana 36110fbc77 update minio/console to latest master 2021-06-25 17:40:10 -07:00
Harshavardhana 3084e9836e upgrade minio/pkg to v1.0.8 2021-06-25 17:17:57 -07:00
Harshavardhana fe49d03fd8 update console dependency for MinIO 2021-06-24 15:39:53 -07:00
Emir İşman 9bf1ac0bb6
[deps]: update jwt-go dependency (#12544)
jwt-go has been renamed to jwt and has a new home. See
https://github.com/dgrijalva/jwt-go/issues/462
2021-06-24 08:41:04 -07:00
Harshavardhana 8f1fe3b761
fix: --console-address when specified endpoints missing (#12534)
Additionally upgrade console dependency for reading
environment variables properly.
2021-06-20 23:04:47 -07:00
Harshavardhana cdeccb5510
feat: Deprecate embedded browser and import console (#12460)
This feature also changes the default port where
the browser is running, now the port has moved
to 9001 and it can be configured with

```
--console-address ":9001"
```
2021-06-17 20:27:04 -07:00
Harshavardhana da74e2f167
move internal/net to pkg/net package (#12505) 2021-06-14 14:54:37 -07:00
Anis Elleuch ba5fb2365c
feat: support of ZIP list/get/head as S3 extension (#12267)
When enabled, it is possible to list/get files
inside a zip file without uncompressing it.

Signed-off-by: Anis Elleuch <anis@min.io>
2021-06-10 08:17:03 -07:00
Harshavardhana e448dbbabf upgrade madmin-go to v1.0.11 2021-06-04 12:51:04 -07:00
Poorna Krishnamoorthy f199afcd6c
tiering: add aws role support for s3 (#12424)
Signed-off-by: Poorna Krishnamoorthy <poorna@minio.io>
2021-06-04 12:47:00 -07:00
Poorna Krishnamoorthy dbea8d2ee0
Add support for existing object replication. (#12109)
Also adding an API to allow resyncing replication when
existing object replication is enabled and the remote target
is entirely lost. With the `mc replicate reset` command, the
objects that are eligible for replication as per the replication
config will be resynced to target if existing object replication
is enabled on the rule.
2021-06-01 19:59:11 -07:00
Harshavardhana 1f262daf6f
rename all remaining packages to internal/ (#12418)
This is to ensure that there are no projects
that try to import `minio/minio/pkg` into
their own repo. Any such common packages should
go to `https://github.com/minio/pkg`
2021-06-01 14:59:40 -07:00
Bala FA 120951d9e9
Refactor health data structure (#11914)
This feature comes with simplified data structures and versioning support.

Signed-off-by: Bala.FA <bala.gluster@gmail.com>
2021-06-01 08:55:49 -07:00