Merge pull request #12112 from amenonsen/vault-stdio

Implement cat-like filtering behaviour for encrypt/decrypt
This commit is contained in:
Toshio Kuratomi 2015-08-27 11:26:48 -07:00
commit 86b2982005
5 changed files with 145 additions and 99 deletions

View file

@ -1,13 +1,13 @@
'\" t
.\" Title: ansible-vault
.\" Author: [see the "AUTHOR" section]
.\" Generator: DocBook XSL Stylesheets v1.78.1 <http://docbook.sf.net/>
.\" Date: 07/28/2015
.\" Generator: DocBook XSL Stylesheets v1.76.1 <http://docbook.sf.net/>
.\" Date: 08/27/2015
.\" Manual: System administration commands
.\" Source: Ansible 2.0.0
.\" Language: English
.\"
.TH "ANSIBLE\-VAULT" "1" "07/28/2015" "Ansible 2\&.0\&.0" "System administration commands"
.TH "ANSIBLE\-VAULT" "1" "08/27/2015" "Ansible 2\&.0\&.0" "System administration commands"
.\" -----------------------------------------------------------------
.\" * Define some portability stuff
.\" -----------------------------------------------------------------
@ -80,19 +80,35 @@ The \fBedit\fR sub\-command is used to modify a file which was previously encryp
This command will decrypt the file to a temporary file and allow you to edit the file, saving it back when done and removing the temporary file\&.
.SH "REKEY"
.sp
*$ ansible\-vault rekey [options] FILE_1 [FILE_2, \&..., FILE_N]
\fB$ ansible\-vault rekey [options] FILE_1 [FILE_2, \&..., FILE_N]\fR
.sp
The \fBrekey\fR command is used to change the password on a vault\-encrypted files\&. This command can update multiple files at once, and will prompt for both the old and new passwords before modifying any data\&.
.SH "ENCRYPT"
.sp
*$ ansible\-vault encrypt [options] FILE_1 [FILE_2, \&..., FILE_N]
\fB$ ansible\-vault encrypt [options] FILE_1 [FILE_2, \&..., FILE_N]\fR
.sp
The \fBencrypt\fR sub\-command is used to encrypt pre\-existing data files\&. As with the \fBrekey\fR command, you can specify multiple files in one command\&.
.sp
Starting with version 2\&.0, the \fBencrypt\fR command accepts an \fB\-\-output FILENAME\fR option to determine where encrypted output is stored\&. With this option, input is read from the (at most one) filename given on the command line; if no input file is given, input is read from stdin\&. Either the input or the output file may be given as \fI\-\fR for stdin and stdout respectively\&. If neither input nor output file is given, the command acts as a filter, reading plaintext from stdin and writing it to stdout\&.
.sp
Thus any of the following invocations can be used:
.sp
\fB$ ansible\-vault encrypt\fR
.sp
\fB$ ansible\-vault encrypt \-\-output OUTFILE\fR
.sp
\fB$ ansible\-vault encrypt INFILE \-\-output OUTFILE\fR
.sp
\fB$ echo secret|ansible\-vault encrypt \-\-output OUTFILE\fR
.sp
Reading from stdin and writing only encrypted output is a good way to prevent sensitive data from ever hitting disk (either interactively or from a script)\&.
.SH "DECRYPT"
.sp
*$ ansible\-vault decrypt [options] FILE_1 [FILE_2, \&..., FILE_N]
\fB$ ansible\-vault decrypt [options] FILE_1 [FILE_2, \&..., FILE_N]\fR
.sp
The \fBdecrypt\fR sub\-command is used to remove all encryption from data files\&. The files will be stored as plain\-text YAML once again, so be sure that you do not run this command on data files with active passwords or other sensitive data\&. In most cases, users will want to use the \fBedit\fR sub\-command to modify the files securely\&.
.sp
As with \fBencrypt\fR, the \fBdecrypt\fR subcommand also accepts the \fB\-\-output FILENAME\fR option to specify where plaintext output is stored, and stdin/stdout is handled as described above\&.
.SH "AUTHOR"
.sp
Ansible was originally written by Michael DeHaan\&. See the AUTHORS file for a complete list of contributors\&.

View file

@ -84,7 +84,7 @@ file, saving it back when done and removing the temporary file.
REKEY
-----
*$ ansible-vault rekey [options] FILE_1 [FILE_2, ..., FILE_N]
*$ ansible-vault rekey [options] FILE_1 [FILE_2, ..., FILE_N]*
The *rekey* command is used to change the password on a vault-encrypted files.
This command can update multiple files at once, and will prompt for both the
@ -93,21 +93,45 @@ old and new passwords before modifying any data.
ENCRYPT
-------
*$ ansible-vault encrypt [options] FILE_1 [FILE_2, ..., FILE_N]
*$ ansible-vault encrypt [options] FILE_1 [FILE_2, ..., FILE_N]*
The *encrypt* sub-command is used to encrypt pre-existing data files. As with the
*rekey* command, you can specify multiple files in one command.
Starting with version 2.0, the *encrypt* command accepts an *--output FILENAME*
option to determine where encrypted output is stored. With this option, input is
read from the (at most one) filename given on the command line; if no input file
is given, input is read from stdin. Either the input or the output file may be
given as '-' for stdin and stdout respectively. If neither input nor output file
is given, the command acts as a filter, reading plaintext from stdin and writing
it to stdout.
Thus any of the following invocations can be used:
*$ ansible-vault encrypt*
*$ ansible-vault encrypt --output OUTFILE*
*$ ansible-vault encrypt INFILE --output OUTFILE*
*$ echo secret|ansible-vault encrypt --output OUTFILE*
Reading from stdin and writing only encrypted output is a good way to prevent
sensitive data from ever hitting disk (either interactively or from a script).
DECRYPT
-------
*$ ansible-vault decrypt [options] FILE_1 [FILE_2, ..., FILE_N]
*$ ansible-vault decrypt [options] FILE_1 [FILE_2, ..., FILE_N]*
The *decrypt* sub-command is used to remove all encryption from data files. The files
will be stored as plain-text YAML once again, so be sure that you do not run this
command on data files with active passwords or other sensitive data. In most cases,
users will want to use the *edit* sub-command to modify the files securely.
As with *encrypt*, the *decrypt* subcommand also accepts the *--output FILENAME*
option to specify where plaintext output is stored, and stdin/stdout is handled
as described above.
AUTHOR
------

View file

@ -260,8 +260,10 @@ class CLI(object):
dest='vault_password_file', help="vault password file", action="callback",
callback=CLI.expand_tilde, type=str)
parser.add_option('--new-vault-password-file',
dest='new_vault_password_file', help="new vault password file for rekey", action="callback",
callback=CLI.expand_tilde, type=str)
dest='new_vault_password_file', help="new vault password file for rekey", action="callback",
callback=CLI.expand_tilde, type=str)
parser.add_option('--output', default=None, dest='output_file',
help='output file name for encrypt or decrypt; use - for stdout')
if subset_opts:

View file

@ -63,8 +63,21 @@ class VaultCLI(CLI):
self.options, self.args = self.parser.parse_args()
self.display.verbosity = self.options.verbosity
if len(self.args) == 0:
raise AnsibleOptionsError("Vault requires at least one filename as a parameter")
can_output = ['encrypt', 'decrypt']
if self.action not in can_output:
if self.options.output_file:
raise AnsibleOptionsError("The --output option can be used only with ansible-vault %s" % '/'.join(can_output))
if len(self.args) == 0:
raise AnsibleOptionsError("Vault requires at least one filename as a parameter")
else:
# This restriction should remain in place until it's possible to
# load multiple YAML records from a single file, or it's too easy
# to create an encrypted file that can't be read back in. But in
# the meanwhile, "cat a b c|ansible-vault encrypt --output x" is
# a workaround.
if self.options.output_file and len(self.args) > 1:
raise AnsibleOptionsError("At most one input file may be used with the --output option")
def run(self):
@ -87,6 +100,28 @@ class VaultCLI(CLI):
self.execute()
def execute_encrypt(self):
if len(self.args) == 0 and sys.stdin.isatty():
self.display.display("Reading plaintext input from stdin", stderr=True)
for f in self.args or ['-']:
self.editor.encrypt_file(f, output_file=self.options.output_file)
if sys.stdout.isatty():
self.display.display("Encryption successful", stderr=True)
def execute_decrypt(self):
if len(self.args) == 0 and sys.stdin.isatty():
self.display.display("Reading ciphertext input from stdin", stderr=True)
for f in self.args or ['-']:
self.editor.decrypt_file(f, output_file=self.options.output_file)
if sys.stdout.isatty():
self.display.display("Decryption successful", stderr=True)
def execute_create(self):
if len(self.args) > 1:
@ -94,13 +129,6 @@ class VaultCLI(CLI):
self.editor.create_file(self.args[0])
def execute_decrypt(self):
for f in self.args:
self.editor.decrypt_file(f)
self.display.display("Decryption successful", stderr=True)
def execute_edit(self):
for f in self.args:
self.editor.edit_file(f)
@ -110,13 +138,6 @@ class VaultCLI(CLI):
for f in self.args:
self.editor.view_file(f)
def execute_encrypt(self):
for f in self.args:
self.editor.encrypt_file(f)
self.display.display("Encryption successful", stderr=True)
def execute_rekey(self):
for f in self.args:
if not (os.path.isfile(f)):

View file

@ -20,6 +20,7 @@ __metaclass__ = type
import os
import shlex
import shutil
import sys
import tempfile
from io import BytesIO
from subprocess import call
@ -130,7 +131,7 @@ class VaultLib:
b_data = to_bytes(data, errors='strict', encoding='utf-8')
if self.is_encrypted(b_data):
raise AnsibleError("data is already encrypted")
raise AnsibleError("input is already encrypted")
if not self.cipher_name or self.cipher_name not in CIPHER_WRITE_WHITELIST:
self.cipher_name = u"AES256"
@ -162,7 +163,7 @@ class VaultLib:
raise AnsibleError("A vault password must be specified to decrypt data")
if not self.is_encrypted(b_data):
raise AnsibleError("data is not encrypted")
raise AnsibleError("input is not encrypted")
# clean out header
b_data = self._split_header(b_data)
@ -227,7 +228,7 @@ class VaultLib:
class VaultEditor:
def __init__(self, password):
self.password = password
self.vault = VaultLib(password)
def _edit_file_helper(self, filename, existing_data=None, force_save=False):
# make sure the umask is set to a sane value
@ -248,11 +249,8 @@ class VaultEditor:
os.remove(tmp_path)
return
# create new vault
this_vault = VaultLib(self.password)
# encrypt new data and write out to tmp
enc_data = this_vault.encrypt(tmpdata)
enc_data = self.vault.encrypt(tmpdata)
self.write_data(enc_data, tmp_path)
# shuffle tmp file into place
@ -261,109 +259,94 @@ class VaultEditor:
# and restore umask
os.umask(old_umask)
def encrypt_file(self, filename, output_file=None):
check_prereqs()
plaintext = self.read_data(filename)
ciphertext = self.vault.encrypt(plaintext)
self.write_data(ciphertext, output_file or filename)
def decrypt_file(self, filename, output_file=None):
check_prereqs()
ciphertext = self.read_data(filename)
plaintext = self.vault.decrypt(ciphertext)
self.write_data(plaintext, output_file or filename)
def create_file(self, filename):
""" create a new encrypted file """
check_prereqs()
# FIXME: If we can raise an error here, we can probably just make it
# behave like edit instead.
if os.path.isfile(filename):
raise AnsibleError("%s exists, please use 'edit' instead" % filename)
# Let the user specify contents and save file
self._edit_file_helper(filename)
def decrypt_file(self, filename):
check_prereqs()
if not os.path.isfile(filename):
raise AnsibleError("%s does not exist" % filename)
tmpdata = self.read_data(filename)
this_vault = VaultLib(self.password)
if this_vault.is_encrypted(tmpdata):
dec_data = this_vault.decrypt(tmpdata)
if dec_data is None:
raise AnsibleError("Decryption failed")
else:
self.write_data(dec_data, filename)
else:
raise AnsibleError("%s is not encrypted" % filename)
def edit_file(self, filename):
check_prereqs()
# decrypt to tmpfile
tmpdata = self.read_data(filename)
this_vault = VaultLib(self.password)
dec_data = this_vault.decrypt(tmpdata)
ciphertext = self.read_data(filename)
plaintext = self.vault.decrypt(ciphertext)
# let the user edit the data and save
if this_vault.cipher_name not in CIPHER_WRITE_WHITELIST:
if self.vault.cipher_name not in CIPHER_WRITE_WHITELIST:
# we want to get rid of files encrypted with the AES cipher
self._edit_file_helper(filename, existing_data=dec_data, force_save=True)
self._edit_file_helper(filename, existing_data=plaintext, force_save=True)
else:
self._edit_file_helper(filename, existing_data=dec_data, force_save=False)
self._edit_file_helper(filename, existing_data=plaintext, force_save=False)
def view_file(self, filename):
check_prereqs()
# decrypt to tmpfile
tmpdata = self.read_data(filename)
this_vault = VaultLib(self.password)
dec_data = this_vault.decrypt(tmpdata)
# FIXME: Why write this to a temporary file at all? It would be safer
# to feed it to the PAGER on stdin.
_, tmp_path = tempfile.mkstemp()
self.write_data(dec_data, tmp_path)
ciphertext = self.read_data(filename)
plaintext = self.vault.decrypt(ciphertext)
self.write_data(plaintext, tmp_path)
# drop the user into pager on the tmp file
call(self._pager_shell_command(tmp_path))
os.remove(tmp_path)
def encrypt_file(self, filename):
check_prereqs()
if not os.path.isfile(filename):
raise AnsibleError("%s does not exist" % filename)
tmpdata = self.read_data(filename)
this_vault = VaultLib(self.password)
if not this_vault.is_encrypted(tmpdata):
enc_data = this_vault.encrypt(tmpdata)
self.write_data(enc_data, filename)
else:
raise AnsibleError("%s is already encrypted" % filename)
def rekey_file(self, filename, new_password):
check_prereqs()
# decrypt
tmpdata = self.read_data(filename)
this_vault = VaultLib(self.password)
dec_data = this_vault.decrypt(tmpdata)
ciphertext = self.read_data(filename)
plaintext = self.vault.decrypt(ciphertext)
# create new vault
new_vault = VaultLib(new_password)
# re-encrypt data and re-write file
enc_data = new_vault.encrypt(dec_data)
self.write_data(enc_data, filename)
new_ciphertext = new_vault.encrypt(plaintext)
self.write_data(new_ciphertext, filename)
def read_data(self, filename):
f = open(filename, "rb")
tmpdata = f.read()
f.close()
return tmpdata
try:
if filename == '-':
data = sys.stdin.read()
else:
with open(filename, "rb") as fh:
data = fh.read()
except Exception as e:
raise AnsibleError(str(e))
return data
def write_data(self, data, filename):
if os.path.isfile(filename):
os.remove(filename)
f = open(filename, "wb")
f.write(to_bytes(data, errors='strict'))
f.close()
bytes = to_bytes(data, errors='strict')
if filename == '-':
sys.stdout.write(bytes)
else:
if os.path.isfile(filename):
os.remove(filename)
with open(filename, "wb") as fh:
fh.write(bytes)
def shuffle_files(self, src, dest):
# overwrite dest with src