Commit graph

5676 commits

Author SHA1 Message Date
Darwin 0f41c185d7 Add support for installing powershell on Amazon Linux (#5461)
- Support for Amazon Linux
- Ensure both tarball installs will reinstall/upgrade
- Clean up comments with incorrect urls
2017-11-15 09:51:52 -08:00
Dongbo Wang c4f0d1c893
Add a test for the powershell hang fix (#5451)
- Add test for the powershell hang fix
- Ignore the 'Sync-PSTags' warning
2017-11-15 09:41:11 -08:00
Keith Hill cef7762d18 Add standard set of vscode workspace setting files (#5457)
* Add standard set of vscode workspace setting files

Add an extensions.json file that will prompt folks to install extensions that are recommended for this workspace (c++, c#, powershell, cmake (editing syntax) and markdown linter.

Add settings.json to start to configure C# syntax, PoweShell trim trailing whitespace and configure the markdown lint extension.

Update launch.json to provide ability to debug PowerShell scripts.

* Clean up grammar

* Instruct VSCode to ensure a final newline.

If this is not wanted for certain file types, we can restrict this option to only specified file types.
2017-11-15 11:18:37 +04:00
Paul Higinbotham 1962c273c6 Fix for PowerShell hang on exit (#5356)
CoreCLR doesn't call finalizer on process exit. PowerShell relies on the CLR finalizer to clean up state on exit. In this case, a Runspace pool was not closed or disposed and any pipeline worker threads created to run concurrent scripts won't end, causing the hang. The same thing can happen if any individual Runspace is created to run a concurrent script and is not closed.

We cannot use the `AppDomain.DomainUnload` event because it's not supported by the default load context. The `AppDomain.ProcessExit` event is also not helpful since it is only called during application exit which means threads already have to be cleaned up.

The fix introduces a static property called `PrimaryRunspace` to `Runspace`. When the PrimaryRunspace is closing it means that the PowerShell session is ending and on exit clean should be performed. The static property `PrimaryRunspace` can only be set once per process.
2017-11-14 14:01:11 -08:00
Steve Lee 89d1d92151 Update governance.md (#5437) 2017-11-14 11:15:31 -08:00
Mark Kraus c832e1687d Add PSTypeName Support for Import-Csv and ConvertFrom-Csv (#5389)
closes #5134

Adds PSTypeName preservation on Import-Csv and ConvertFrom-Csv
2017-11-14 11:15:18 -08:00
Steve Lee 83e1d64669 Enable install-powershell.ps1 to update the current running powershell (#5429)
1. Remove `[validate]` attribute from parameter so this script can be invoked directly from the web. This doesn't have any negative impact as the $Destination parameter will have a default value if null or empty.
2. Move removal of destination folder later as installing the package requires package management module and archive module (on Windows)
3. On Windows, because files have open handles when run from existing powershell-daily install, I rename the existing files and copy over the new ones.  User needs to exit and restart `pwsh` to take effect (similar to macOS/Linux where you have to exit and restart anyways)
2017-11-14 10:45:08 -08:00
Travis Plunk 31f7639552
Add compliance label description (#5355) 2017-11-14 10:37:24 -08:00
Travis Plunk 1d64cad068
Signing fixes (#5439)
* Add two files that need to be signed
* make sure to set PSModuleRestore to true when expanding a signed build because we run PSModuleRestore at that point.
* suppress output of CmdLets which are noisy
2017-11-14 10:30:17 -08:00
Mark Kraus 87bcd4132a Fix Single Value JSON null in Invoke-RestMethod (#5338)
* [Feature] Fix Single Value JSON null in Invoke-WebRequest

* [Feature] Switch from Regex to JToken.Parse()

* [Feature] Address PR Feedback: use `n
2017-11-14 07:53:43 +04:00
Dongbo Wang da0b384b2a Try fixing the 'brew update' issue in bootstrap script (#5400)
* Try fixing the 'brew update' issue in bootstrap script

* Add a log message

* Add a comment
2017-11-13 12:21:53 -08:00
Mark Kraus ecf0f8c151 Add SslProtocol Support to WebCmdlets (#5329)
closes #2662

This feature adds the ability to restrict the SSL/TLS protocol used when making the web request. In 5.1 the user could make use of .NET API's to enforce this on the Web Cmdlets. With the move to HttpClient in PowerShell Core, those APIs have no impact. The user still has requirements to ensure specific protocols are used.

The public enum WebSslProtocol is added as a wrapper to the underlying SslProtocols enum. Neither it nor SecurityProtocolType can be used because Ssl3 and Ssl2 are not supported by HttpClientHandler.SslProtocols. While it may not be intuitive to a PowerShell user to use -bor or "Tls, Tls11" to set multiple options, the general use case for this will be a single protocol.

Adds -SslProtocol parameter to Web Cmdlets
Adds WebSslProtocol Enum to support limited subset of SslProtocol enum supported by HttpClientHandler
Adds TLS 1.1 and TLS 1.0 listening ports to WebListener
2017-11-13 10:45:46 -08:00
Dan Travison d5a974ffd4 Add dependency nuget instructions for PowerShell.Core.Instrumentation resource binary (#5396)
* Add dependency nuget packaging instructions for PowerShell.Core.Instrumentation ETW resource binary.

* Fix markdown MD031 Fenced code blocks errors.

* Fix MD022 Headers should be surrounded by blank lines

* Add a note about Authenticode Dual-signing requirement.

* PR Feedback.

* Set version to Beta.10
2017-11-13 10:40:31 -08:00
Andrew 06d3114763 Automation for PS package validation using Docker containers (#5401)
This automation is used for basic validation of PS packages during a release;
Covered configs:

ubuntu14.04
ubuntu16.04
ubuntu17.04
debian.8
debian.9
centos7
fedora25
fedora26
kalilinux
opensuse42.2
During the process Docker files are filled out and executed with Docker build;
During the build PS packages are downloaded onto Docker containers, installed and selected Pester tests from PowerShell Github repo are executed.
2017-11-13 10:25:09 -08:00
Travis Plunk 36fac11f29 Allow packaging from a zip package to allow for signing (#5418)
- Include a serialized version of PSOptions in an includesymbols zip
- Add a function which will create a zip package from the expanded includesymbols zip and a folder of signed files
- Add a function to restore an includesymbols zip as a build and populated PSOptions with the options
2017-11-13 10:10:51 -08:00
Mark Kraus 592cd384e8 Fix spelling in web cmdlet errors (#5427) 2017-11-12 17:48:40 -08:00
Steve Lee 0f223c00d2 Move the 'rcedit' step to build rather than packaging (#5404) 2017-11-11 10:32:10 -08:00
Dongbo Wang 2f399e233c
Enhance 'install-powershell.ps1' to work on Linux/MacOS (#5411)
Enhance 'install-powershell.ps1' to work on Linux and macOS too. A user can install release version or daily powershell core on Linux and macOS.
When `-AddToPath` is specified:
- On Windows, add the absolute destination path to the 'User' scope environment variable 'Path';
- On Linux, make the symlink '/usr/bin/pwsh' points to "$Destination/pwsh";
- On MacOS, make the symlink '/usr/local/bin/pwsh' points to "$Destination/pwsh".
2017-11-10 17:54:24 -08:00
Steve Lee 54f34a0bac Support '[package]' tag in PR CI and fix nightly build on macOS (#5410)
- Support '[package]' tag in PR CI to validate packaging changes in PR CI runs.
  When using '[package]' tag, regular CI tests will run and packaging steps will also run.
- Fix nightly build on macOS.
2017-11-10 17:28:55 -08:00
Travis Plunk b1d312317c
make name a dynamic parameter (#5415) 2017-11-10 15:28:38 -08:00
Ilya c86f243ca0 Make Import-Csv support CR, LF and CRLF as line delimiters (#5363)
With the fix, `Import-Csv` support CR (\r), LF (\n), CRLF (\r\n) as line delimiters.
2017-11-10 14:02:45 -08:00
Chunqing Chen 2be13a623e Disable cmdlets that are not supported under unix system (#5083)
* permanently remove cmdlets that are not supported under Unix and move tests to defaultcommands.tests.ps1
2017-11-10 10:36:08 -08:00
Mark Kraus f5f3fab1ea Clean Up Full CLR Code From Web Cmdlets (#5376) 2017-11-10 22:00:11 +04:00
Himura Kazuto 6530b779d9 Multiple improvements by CodeRush Static Analysis (#5132) 2017-11-10 09:03:34 -08:00
Dongbo Wang affcc40a52
Add 'install-powershell.ps1' to install powershell core on windows (#5383)
- Add install-powershell.ps1 to install powershell core packages on windows.
- Update Start-PSBootStrap to check and install the latest PSCore package on Windows.
2017-11-10 08:28:02 -08:00
Mark Kraus ee7edb7427 Add error on Legacy Credential over non-HTTPS for Web Cmdlets (#5402)
- Add an error when a user tries to use `-Credential` (legacy usage without `-Authentication`) or `-UseDefaultCredentials` over a non-HTTPS URI
- User can bypass error with `-AllowUnencryptedAuthentication`
- `-UseDefaultCredentials` can only be reliably tested on Windows as support on other platforms depends on a Kerberos infrastructure. 
- Add `/Auth/` tests to WebListener for challenge authentication Basic, Negotiate, and NTLM

Incidentally, this increases test coverage for the web cmdlets as `-Credential` and `-UseDefaultCredentials` were not being tested.
2017-11-10 08:11:57 -08:00
Ilya 5fd1b4759c
Don't write an error if file already unblocked (Unblock-File) (#5362) 2017-11-10 18:14:08 +04:00
Dongbo Wang 55f6827aa9
Revive 'download.sh' and make it redirect to 'install-powershell.sh' (#5386) 2017-11-09 22:58:53 -08:00
Travis Plunk 1a5134f9e4
Add signing manifest and script to update it with production values (#5397)
* Add signing configuration file
* add updatesigning.ps1
2017-11-09 17:47:03 -08:00
Keith Hill 34600c4100 Update Find-Dotnet to find dotnet with compat SDK (#5341)
Fix #5260 - with this fix folks shouldn't hit the version mismatch check.

The approach is to see if the dotnet in the current PATH has a compatible SDK.  Folks will have a globall installed dotnet if they've installed VS, VSCode C# ext or have  installed the .NET Core SDK.  This verion may not have the SDK required by PSCore.  And the global cannot see user local dotnet SDK installs.  So if the global dotnet doesn't have the right SDK, we prepend the path to the user's local dotnet dir.

Also, updateed $dotnetCLIRequiredVersion to read its value from global.json so there is one less source of the truth (for the SDK version).
2017-11-09 16:28:39 -08:00
Paul Higinbotham 73188ce27a Porting random hang fixes (#5258) 2017-11-09 16:22:39 -08:00
Steve Lee e0ee555ad6 fix help function to not pipe to more if objects are returned instead of help text (#5395) 2017-11-09 16:21:49 -08:00
Steve Lee 8b42e1748e Add ability to produce tar.gz pkg for Raspbian (#5387) 2017-11-09 10:48:55 -08:00
Dan Travison 3bfae4f1dd Rename/Update PowerShell ETW manifest to remove the Windows PowerShell dependency. (#5360)
- Build PowerShell.Core.Instrumentation.dll - Resource-only binary for the ETW resources.
- Create a registration script for registering/unregistering the ETW provider.
2017-11-09 09:23:00 -08:00
Steve Lee b1947313c1 Revert refactoring changes that broke remoting to Windows PowerShell 5.1 (#5321)
- Revert refactoring changes so that it doesn't break remoting to Windows PowerShell 5.1
- Fix the rest of the supported types for remoting
- Put remoting sensitive private members in a descriptive region
2017-11-09 09:13:34 -08:00
Dan Travison 7a78fec180 Exclude EventResource.resx from SMAResources.Tests.ps1 (#5379)
Fix a test failure.
2017-11-09 07:52:01 -08:00
Ilya c4269cb6be Make SemanticVersion compatible with SemVer 2.0 (#5037)
Made 'SemanticVersion' compatible with SymVer 2.0 http://semver.org/ (p.10)
- Fix comparisons
- Refactor and add more tests
2017-11-09 07:44:58 -08:00
Travis Plunk ab24ac2b8a Use simplified names for compliance folders (#5388) 2017-11-09 07:29:02 -08:00
Avi Wollman b3940fe15b Correct the Fedora documentation (#5384)
Correction to the Fedora documentation: "Fedora machine" instead of "Red Hat Enterprise Linux machine"
2017-11-08 13:51:55 -08:00
Steve Lee 5f5407595d replace the word hang with something more appropriate (#5358) 2017-11-08 13:47:12 -08:00
Dongbo Wang 906d0a5576
Minor update to FAQ.md (#5366) 2017-11-08 13:41:53 -08:00
Steve Lee 1721f348a5 Update instructions to build on VSCode based on pwsh (#5368) 2017-11-08 07:40:40 +04:00
James Truher [MSFT] fe3e44f305 Change $OutputEncoding to be utf8 without BOM rather than ASCII (#5369) 2017-11-07 14:49:14 -08:00
Travis Plunk 610c70d952 set expected binaries and variable name for folder for symbols build (#5357) 2017-11-07 12:09:34 -08:00
Jason Shirk 8391b9e155 Rework passing array literal to native commands (#5301) 2017-11-06 17:05:26 -08:00
Greg Zimmerman a674c518b1 Enable macOS launcher (#5291) 2017-11-06 10:44:20 -08:00
bergmeister 6b700bbcc6 Update Wix toolset download link to newer version 3.11 (#5339)
This makes it also future proof since codeplex is going to get deprecated.
2017-11-06 10:32:00 -08:00
Steve Lee bbbaf287ce Change all links in README.md to absolute as it is being reused in other places outside of GitHub (#5354) 2017-11-06 10:29:49 -08:00
Keith Hill c781959232 Display full help with 'help' function (#5195)
Display full help with 'help' function
2017-11-06 10:19:39 -08:00
Dan Travison 2670917981 Redirect ETW logging to Syslog on Linux (#5144)
This PR is divided into the following areas:
1. Add/Rename the PowerShell/Windows ETW manifest to the repo and change both the provider id (guid) and name. See #4939.
The manifest is at tools/resxgen/PowerShell-Core-Instrumentation.man

2. Generate a resx file containing the string resources needed for logging ETW events to syslog.
This is accomplished by tools\resxgen\resxgen.psm1 and resxgen.ps1. The tool generates two files

A resx file containing string resources for each message string from the manifest. This is generated in the System.Management.Automation\gen directory.
A C# class (EventResource) that provides the mapping between the integer event id and the associated string resource name. The file is generated in the System.Management.Automation\CoreCLR directory with a compile-time condition of UNIX
NOTE: The EventResource.cs class generated by resgen is explicitly ignored in the csproj file; it is not used.

3. SMA\utils\tracing\PSSysLogProvider.cs
Implements the abstract LogProvider class and is the syslog equivalent of PSEtwLogProvider. The class contains a number of logical methods for logging lifecycle, health, and normal events.

4. SMA\utils\tracing\SysLogProvider.cs
This is the Syslog equivalent of ETW's log provider class and implements a Log method versus ETW's EventWrite. It is also responsible for resolving event ids to resource names and performing the Syslog call. There is a large comment block in the class XML doc that describes the types of log output it produces.

5. SMA\utils\tracing\PSEtwLog.cs
PowerShell's current implementation is tightly coupled to this class; with code calling it directly for all events. To simplify integration of syslog, I updated the class to create an instance of PSSysLogProvider on Linux and removed the Linux-specific stub file.

6. SMA\engine\PropertyAccessor.cs
This class provides a wrapper around PowerShellProperties.json and has been extended to have Unix-specific assessors for configuring logging. Note that the file is expected to be in the $PSHOME directory to ensure SxS.

Currently, there are four configuration properties:

- LogIdentity - the string identifier for the source application. This defaults to 'powershell' and can be configured to enable distinguishing between side-by-side installations.
- LogLevel - configures the tracing level (log level). Informational is the defauilt.
- LogChannels - used to enable operational and analytic logging. Operational is the default.
- LogKeywords - used to configure enabling tracing by keyword. All keywords other than `UseAlwaysAnalytic` are enabled by default.

NOTE: This will likely change. PowerShell sometimes confuses the analytic channel with this keyword and sends logging to the wrong channel. Once this is cleared up, `UseAlwaysAnalytic` and `UseAlwaysOperational` keywords will likely be removed.

Additional Notes:

1. The current implementation writes directly to syslog and writing to a separate log file is still pending.
2. The generated class and resx are not part of the build; instead, it is expected that Resxgen should be run when events are added to the manifest. To fully automate the process, resxgen will need to be updated to generate the other dependent enums such as 'PSChannel', 'PSEventId', 'PSTask', 'PSOpcode', etc. You will see parsing logic in resxgen.psm1 to prepare for that but it is not enabled at this point.
4. Documentation is pending that documents the format of the syslog output as well as associated configuration.
5. As mentioned at the start, tests are pending
2017-11-06 08:32:29 -08:00