Commit graph

3786 commits

Author SHA1 Message Date
Krishnan Parthasarathi d0963974a5
pkg/lifecycle: Add SetPredictionHeaders method (#12755)
This method is used to add expected expiration and transition time 
for an object in GET/HEAD Object response headers.

Also fixed bugs in lifecycle.PredictTransitionTime and
getLifecycleTransitionTier in handling current and 
non-current versions.
2021-07-20 17:36:55 -07:00
Krishnan Parthasarathi 6ea083d197
Add deployment-id and source bucket to transitioned object name (#12693)
This allows remote bucket admin to identify the origin of transitioned
objects by simply inspecting the object prefixes.

e.g let's take a remote tier TIER-1 pointing to a remote bucket (prefix)
testbucket/testprefix-1. The remote bucket admin can list all transitioned objects
from a MinIO deployment identified by '2e78e906-1c5d-4f94-8689-9df44cafde39' and
source bucket 'mybucket' like so,

```
$ ./mc ls -r minio-tier-target/testbucket/testprefix-1/2e78e906-1c5d-4f94-8689-9df44cafde39/mybucket/
[2021-07-12 17:15:50 PDT]   160B 48/fb/48fbc0e6-3a73-458b-9337-8e722c619ca4
[2021-07-12 16:58:46 PDT]   160B 7d/1c/7d1c96bd-031a-48d4-99ea-b1304e870830
```
2021-07-20 10:49:52 -07:00
Harshavardhana ab7584eeac update to latest console fs/gateway 2021-07-19 17:49:54 -07:00
sakkiii 69e0faa278
add more XSS HTTP headers (#12256) 2021-07-19 16:05:02 -07:00
Shireesh Anjal fa98014bbe
fix: hostname inside --console-address not anonymized (#12743)
In case of non-distributed setup, if the server start command contains a
`--console-address` flag and its value contains a hostname, it is not
getting anonymized.

Fixed by replacing the console host also with `server1`
2021-07-19 15:16:26 -07:00
Harshavardhana 27e07e80bc
fix: accountInfo() error FS mode (#12734) 2021-07-17 01:17:35 -07:00
Harshavardhana b13bca85d0
send backendInfo as part AccountInfo (#12733)
This is needed for console UI to change
the UI behavior for different modes of
operation.
2021-07-16 14:37:06 -07:00
Krishnan Parthasarathi 29eea52e14
Skip transitioning of object versions if inlined (#12705) 2021-07-16 09:38:27 -07:00
Anis Elleuch b0b4696a64
heal: Add MRF metrics to background heal API response (#12398)
This commit gathers MRF metrics from 
all nodes in a cluster and return it to the caller. This will show information about the 
number of objects in the MRF queues 
waiting to be healed.
2021-07-15 22:32:06 -07:00
Aditya Manthramurthy eae9c2f65b
Add changes to ensure session policy is enforced in LDAP STS (#12716)
- Bonus: Fix bug in webidentity sts that doesnt parse session policy correctly.

- update ldap.go to support session policy argument
2021-07-15 15:27:34 -07:00
Poorna Krishnamoorthy a6ec405443
fix: UpdateObjectVersion should compare versionID through versions (#12726)
fixes #12703
2021-07-15 15:01:59 -07:00
Harshavardhana 9516587a6c update console to master branch
with new fixes

- improve download behavior
- avoid response timeouts
2021-07-15 14:44:18 -07:00
Harshavardhana 017722cf85
fix: propagate service account deletes properly (#12717)
service account deletes were not propagating
to remote peers, fix this.
2021-07-14 21:28:53 -07:00
Harshavardhana 10fc30a989
fix crash when config is not properly initialized (#12714)
fixes #12709
2021-07-14 18:27:25 -07:00
Lazzaretti e8cbfa7af2
fix: rename error message for MINIO_BROWSER_REDIRECT_URL (#12700)
rename the error message for MINIO_BROWSER_REDIRECT_URL to match the field name
2021-07-14 12:38:25 -07:00
Anis Elleuch b8f95fb3d4
fix: Use correct replication status in replication healing (#12711)
In case of replication healing, we always store completed status in the
object metadata, which is wrong because replication could fail in the
further retries.
2021-07-14 09:58:46 -07:00
Anis Elleuch a4b8928660
Move prefix usage in admin AccountInfo API (#12710) 2021-07-14 08:51:10 -07:00
Shireesh Anjal 4d6d4244f1
Anonymize hosts in subnet health report (#12627)
Ensure that hostnames / ip addresses are not printed in the subnet
health report. Anonymize them by replacing them with `servern` where `n`
represents the position of the server in the pool.

This is done by building a `host anonymizer` map that maps every
possible value containing the host e.g. host, host:port,
http://host:port, etc to the corresponding anonymized name and using
this map to replace the values at the time of health report generation.

A different logic is used to anonymize host names in the `procinfo`
data, as the host names are part of an ellipses pattern in the process
start command. Here we just replace the prefix/suffix of the ellipses
pattern with their hashes.
2021-07-14 00:23:22 -07:00
ebozduman 83c37a44b6
'starts-with' support for 'success_action_status' (#12698) 2021-07-13 13:33:03 -07:00
Harshavardhana 9dae5a7c85
fix: only redirect for GET/HEAD requests (#12702)
fixes #12701
2021-07-13 11:25:08 -07:00
Anis Elleuch aa78505181
Add prefixes usage in Accounting Usage Info (#12687) 2021-07-13 10:42:11 -07:00
Harshavardhana e316873f84
feat: Add support for kakfa audit logger target (#12678) 2021-07-13 09:39:13 -07:00
Harshavardhana 559d075627
fix: properly send group unset policy (#12694)
In distributed setups `mc admin group unset`
was not correctly working due to incorrect
handling of query parameters between peers.
2021-07-13 09:32:35 -07:00
Harshavardhana a655e7f820
fix: use browser redirect URL for IDP callback (#12689)
if browser_redirect_url is set use that for IDP callback
automatically, if we do not have to set REDIRECT_URI
for OpenID callback URL.
2021-07-12 15:21:07 -07:00
Klaus Post 3201bae773
feat: gzip S3/Admin API responses whenever applicable (#12631)
Gzip responses if appropriate, except GetObject requests.
List reponses has an almost 10:1 compression ratio with no
measurable slowdown (in fact it seems a bit faster).
2021-07-12 10:49:50 -07:00
Anis Elleuch 8625cfdfae
update madmin-go dependency, update related code (#12669) 2021-07-12 10:16:10 -07:00
Harshavardhana d86ef6256d fix: ldap:username variable substitution in policies 2021-07-11 18:39:06 -07:00
Harshavardhana cd36019450
fix: allow STS credentials with dynamic policies (#12681)
- ParentUser for OIDC auth changed to `openid:`
  instead of `jwt:` to avoid clashes with variable
  substitution

- Do not pass in random parents into IsAllowed()
  policy evaluation as it can change the behavior
  of looking for correct policies underneath.

fixes #12676
fixes #12680
2021-07-11 17:39:52 -07:00
Harshavardhana 931f73f59a
fix: add IAM dummy store for gateway operations (#12670)
with console addition users cannot login with
root credentials without etcd persistent layer,
allow a dummy store such that such functionalities
can be supported when running as non-persistent
manner, this enables all calls and operations.
2021-07-10 08:32:52 -07:00
Anis Elleuch 9be040dd14
Add option in readDir to enable symlink following of dirs (#12668) 2021-07-09 16:20:51 -07:00
Harshavardhana da0fd5f056
fix: allow customizing console redirection (#12665)
MinIO might be running inside proxies, and
console while being on another port might not be
reachable on a specific port behind such proxies.

For such scenarios customize the redirect URL
such that console can be redirected to correct
proxy endpoint instead.

fixes #12661
2021-07-09 14:27:09 -07:00
Anis Elleuch c8cf4c5eb8
Fix force bucket deletion in distributed mode (#12659)
storageRESTForceDelete is not always passed by client, use r.URL().Query
instead.

Co-authored-by: Anis Elleuch <anis@min.io>
2021-07-09 13:29:38 -07:00
Klaus Post d6a2fe02d3
Add admin file inspector (#12635)
Download files from *any* bucket/path as an encrypted zip file.

The key is included in the response but can be separated so zip 
and the key doesn't have to be sent on the same channel.

Requires https://github.com/minio/pkg/pull/6
2021-07-09 11:29:16 -07:00
Harshavardhana 28adb29db3
feat: Add support to poll users on external SSO (#12592)
Additional support for vendor-specific admin API
integrations for OpenID, to ensure validity of
credentials on MinIO.

Every 5minutes check for validity of credentials
on MinIO with vendor specific IDP.
2021-07-09 11:17:21 -07:00
Harshavardhana b79cdc1611
fix: allow gateway to work with root credentials (#12655) 2021-07-09 10:35:09 -07:00
Harshavardhana 84a64a7e47
fix: accountInfo should use parentUser (#12652)
parentUser is needed to make sure that
dynamic variables in policy work properly.

fixes #12651
2021-07-08 12:43:25 -07:00
Harshavardhana affee27b05
fix: speed up erasure code upgrade checks (#12626)
DiskInfo() calls can stagger and wait if run
serially timing out 10secs per drive, to avoid
this lets check DiskInfo in parallel to avoid
delays when nodes get disconnected.
2021-07-08 01:04:37 -07:00
Harshavardhana 2ce60d772b
fix: update README.md for new release (#12637)
rename all 'docker run' commands to 'podman run'

fixes #12633
2021-07-07 18:07:29 -07:00
Harshavardhana c99d399d09
fix: handle redirects for specific resources (#12629) 2021-07-07 12:04:16 -07:00
Harshavardhana 25f55d6051 fix: order of args ListObjectsV2 S3 gateway 2021-07-06 13:44:14 -07:00
Harshavardhana 039978640f
fix: honor system umask for file creates (#12601)
use 0666 os.FileMode to honor system umask
2021-07-06 12:54:16 -07:00
Harshavardhana 6503c6ac21 fix: support startAfter with S3 gateway
fixes #12604
2021-07-05 18:44:04 -07:00
Klaus Post 05aebc52c2
feat: Implement listing version 3.0 (#12605)
Co-authored-by: Harshavardhana <harsha@minio.io>
2021-07-05 15:34:41 -07:00
Klaus Post a7e2a1a38b
fix: two different scanner update races (#12615) 2021-07-02 11:19:56 -07:00
Klaus Post f706671568
fix: multi-way list entry resolution (#12617)
Fixes brought forward from https://github.com/minio/minio/pull/12605

Fixes resolution when an object is in prefix of another and one zone returns the directory and another the object.

Fixes resolution on single entries that arrive first, so resolution doesn't depend on order.
2021-07-02 09:54:00 -07:00
Harshavardhana 4f6c74a257
simplify audit logging for replication and ILM (#12610)
auditLog should be attempted right before the
return of the function and not multiple times
per function, this ensures that we only trigger
it once per function call.
2021-07-01 14:02:44 -07:00
Krishnan Parthasarathi a1df230518
Add a 'free' version to track deletion of tiered object content (#12470) 2021-06-30 19:32:07 -07:00
Aditya Manthramurthy dc6958b6a1
fix: enhance openid claim missing error (#12608)
The error implies an expected claim is missing even when 
the claim is present. Added an additional error message 
to clarify the problem.
2021-06-30 17:11:23 -07:00
Harshavardhana 5a33d3f05a fix: set the correct IDP salt/passphrase 2021-06-30 16:45:52 -07:00
Harshavardhana 4781e7580b
add support for customizing redirect_uri for IDP (#12607) 2021-06-30 16:08:20 -07:00