diff --git a/.github/CODE_OF_CONDUCT.md b/.github/CODE_OF_CONDUCT.md new file mode 100644 index 00000000..686e5e7a --- /dev/null +++ b/.github/CODE_OF_CONDUCT.md @@ -0,0 +1,10 @@ +# Microsoft Open Source Code of Conduct + +This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). + +Resources: + +- [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/) +- [Microsoft Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) +- Contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with questions or concerns +- Employees can reach out at [aka.ms/opensource/moderation-support](https://aka.ms/opensource/moderation-support) diff --git a/.github/SECURITY.md b/.github/SECURITY.md new file mode 100644 index 00000000..f941d308 --- /dev/null +++ b/.github/SECURITY.md @@ -0,0 +1,41 @@ + + +## Security + +Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/Microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet), [Xamarin](https://github.com/xamarin) and [PowerShell](https://github.com/PowerShell). + +If you believe you have found a security vulnerability in any Microsoft-owned repository that meets [Microsoft's definition of a security vulnerability](https://aka.ms/security.md/definition), please report it to us as described below. + +## Reporting Security Issues + +**Please do not report security vulnerabilities through public GitHub issues.** + +Instead, please report them to the Microsoft Security Response Center (MSRC) at [https://msrc.microsoft.com/create-report](https://aka.ms/security.md/msrc/create-report). + +If you prefer to submit without logging in, send email to [secure@microsoft.com](mailto:secure@microsoft.com). If possible, encrypt your message with our PGP key; please download it from the [Microsoft Security Response Center PGP Key page](https://aka.ms/security.md/msrc/pgp). + +You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://www.microsoft.com/msrc). + +Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue: + + * Type of issue (e.g. buffer overflow, SQL injection, cross-site scripting, etc.) + * Full paths of source file(s) related to the manifestation of the issue + * The location of the affected source code (tag/branch/commit or direct URL) + * Any special configuration required to reproduce the issue + * Step-by-step instructions to reproduce the issue + * Proof-of-concept or exploit code (if possible) + * Impact of the issue, including how an attacker might exploit the issue + +This information will help us triage your report more quickly. + +If you are reporting for a bug bounty, more complete reports can contribute to a higher bounty award. Please visit our [Microsoft Bug Bounty Program](https://aka.ms/security.md/msrc/bounty) page for more details about our active programs. + +## Preferred Languages + +We prefer all communications to be in English. + +## Policy + +Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://aka.ms/security.md/cvd). + + diff --git a/.gitignore b/.gitignore new file mode 100644 index 00000000..03eee07d --- /dev/null +++ b/.gitignore @@ -0,0 +1,2 @@ +.vscode/styles +.vale/ diff --git a/.markdownlint-cli2.yaml b/.markdownlint-cli2.yaml new file mode 100644 index 00000000..93bc09d9 --- /dev/null +++ b/.markdownlint-cli2.yaml @@ -0,0 +1,36 @@ +# Rule definitions live in .markdownlint.json + +# Include a custom rule package +# customRules: +# - markdownlint-rule-titlecase + +# Fix any fixable errors +fix: true + +# Define a custom front matter pattern +# frontMatter: (^---\s*$[^]*?^---\s*$)(\r\n|\r|\n|$) + +# Define glob expressions to use (only valid at root) +# globs: +# - "!*bout.md" + +# Define glob expressions to ignore +ignores: + - .vscode + - assets + - tests + - tools + +# Use a plugin to recognize math +# markdownItPlugins: +# - - "@iktakahiro/markdown-it-katex" + +# Disable inline config comments +noInlineConfig: false + +# Disable progress on stdout (only valid at root) +noProgress: true + +# Use a specific formatter (only valid at root) +# outputFormatters: +# - [markdownlint-cli2-formatter-default] diff --git a/.markdownlint.yaml b/.markdownlint.yaml new file mode 100644 index 00000000..dd99708e --- /dev/null +++ b/.markdownlint.yaml @@ -0,0 +1,103 @@ +MD001: true # header-increment +# MD002 # first-header-h1 - Superceded by MD041 +MD003: # header-style + style: atx +MD004: # ul-style + style: dash +MD005: true # list-indent +# MD006 # ul-start-left - Superceded by MD007's start_indented option +MD007: # ul-indent + indent: 2 + start_indented: false +# MD008 # Removed from linter; used to specify indentation for ul +MD009: # no-trailing-spaces + br_spaces: 2 + strict: true +MD010: true # no-hard-tabs +MD011: true # no-reversed-links +MD012: true # no-multiple-blanks +MD013: # line-length + code_block_line_length: 90 + code_blocks: true + heading_line_length: 100 + headings: true + line_length: 100 + stern: true + tables: false +MD014: true # commands-show-output +# MD015 # "Use of non-atx style headers" - Removed from linter, replaced by MD003 +# MD016 # "Use of non-closed-atx style headers" - Removed from linter, replaced by MD003 +# MD017 # "Use of non-setext style headers" - Removed from linter, replaced by MD003 +MD018: true # no-missing-space-atx +MD019: true # no-multiple-space-atx +MD020: true # no-missing-space-closed-atx +MD021: true # no-multiple-space-closed-atx +MD022: true # blanks-around-headers +MD023: true # header-start-left +MD024: # no-duplicate-header + siblings_only: true +MD025: # single-h1 + front_matter_title: '' + level: 1 +MD026: # no-trailing-punctuation + punctuation: '.,;:!。，；：！？' +MD027: true # no-multiple-space-blockquote +MD028: true # no-blanks-blockquote +MD029: # ol-prefix + style: one +MD030: true # list-marker-space +MD031: true # blanks-around-fences +MD032: true # blanks-around-lists +MD033: # no-inline-html + allowed_elements: + - a + - br + - code + - kbd + - li + - properties + - sup + - tags + - ul +MD034: true # no-bare-urls +MD035: # hr-style + style: '---' +MD036: true # no-emphasis-as-header +MD037: true # no-space-in-emphasis +MD038: true # no-space-in-code +MD039: true # no-space-in-links +MD040: false # fenced-code-language +MD041: false # first-line-h1 +MD042: true # no-empty-links +MD043: false # required-headers +MD044: # proper-names + code_blocks: false + names: + - PowerShell + - IntelliSense + - Authenticode + - CentOS + - Contoso + - CoreOS + - Debian + - Ubuntu + - openSUSE + - RHEL + - JavaScript + - .NET + - NuGet + - VS Code + - Newtonsoft +MD045: true # no-alt-text +MD046: # code-block-style + style: fenced +MD047: true # single-trailing-newline +MD048: # code-fence-style + style: backtick +MD049: # emphasis-style + style: underscore +MD050: # strong-style + style: asterisk +MD051: true # link-fragments +MD052: true # reference-links-images +MD053: true # link-image-reference-definitions diff --git a/.vale.ini b/.vale.ini new file mode 100644 index 00000000..11df0719 --- /dev/null +++ b/.vale.ini @@ -0,0 +1,10 @@ +StylesPath = .vscode/styles +Packages = https://microsoft.github.io/Documentarian/packages/vale/PowerShell-Docs.zip +MinAlertLevel = suggestion + +[*] +BasedOnStyles = PowerShell-Docs +Vale.Spelling = NO # We use cspell + +[*.md] +BasedOnStyles = PowerShell-Docs diff --git a/.vscode/cSpell.json b/.vscode/cSpell.json new file mode 100644 index 00000000..09a86bf2 --- /dev/null +++ b/.vscode/cSpell.json @@ -0,0 +1,6 @@ +{ + "import": [ + "./cspell/psdocs/cspell.yaml", + "./cspell/blog/cspell.yaml" + ] +} diff --git a/.vscode/cspell/blog/cspell.yaml b/.vscode/cspell/blog/cspell.yaml new file mode 100644 index 00000000..044c967a --- /dev/null +++ b/.vscode/cspell/blog/cspell.yaml @@ -0,0 +1,26 @@ +name: blog +description: > + This cSpell configuration is for terminology used on the Blog. Only add to the dictionary defined + here if the terms aren't for general PowerShell. If they apply to PowerShell more broadly, + contribute those changes back to the psdocs configuration and dictionaries in the + MicrosoftDocs/PowerShell-Docs repository, not here. + +dictionaryDefinitions: + - name: blog-terms + description: > + Dictionary of common terms used in the blog.. Add entries to this dictionary for words used + in blog posts but not broader PowerShell or other documentation. + path: ./dictionaries/blog-terms.txt + +# These settings are applied to combinations of language (file type) and locale. For any given file +# and locale, all matching dictionaries are applied. +languageSettings: + # Any Markdown file + - languageId: markdown + locale: '*' + dictionaries: + - blog-terms + - languageId: yaml + locale: '*' + dictionaries: + - blog-terms diff --git a/.vscode/cspell/blog/dictionaries/blog-terms.txt b/.vscode/cspell/blog/dictionaries/blog-terms.txt new file mode 100644 index 00000000..e69de29b diff --git a/.vscode/cspell/psdocs/cspell.yaml b/.vscode/cspell/psdocs/cspell.yaml new file mode 100644 index 00000000..370b6eea --- /dev/null +++ b/.vscode/cspell/psdocs/cspell.yaml @@ -0,0 +1,158 @@ +enabled: true +description: > + This configuration defines the default spellcheck settings for all PowerShell documentation. Where + needed, other projects and subfolders can extend or override these defaults. + +# Ensure that any comments in code files to controll cSpell are correct. +validateDirectives: true + +# These apply to all files unless otherwise specified. They're defined in NPM modules that are +# available by default with the extension. +dictionaries: + - azureTerms + - companies + - filetypes + - misc + - powershell + - softwareTerms + +# These are locally defined. They must be specified for the document type they're used in. +dictionaryDefinitions: + - name: externalCommands + description: > + Dictionary of common commands external to PowerShell. Add entries to this dictionary for + commands, services, and script keywords that are referenced in documentation but are not + specific to or included in PowerShell. + path: ./dictionaries/externalCommands.txt + - name: fictionalCorps + description: > + Dictionary of fictional company names used in documentation. Add entries to this dictionary + when using a valid but nonexistant fictional company or organization. + path: ./dictionaries/fictionalCorps.txt + - name: fileExtensions + description: > + Dictionary of file extensions referenced in documentation. Add entries to this dictionary + when a valid file extension is marked as an unknown spelling. + path: ./dictionaries/fileExtensions.txt + - name: psdocs + description: > + General PowerShell documentation dictionary. Add entries to this dictionary for PowerShell + concepts, terms, or other names. Consider submitting them to the upstream PowerShell + dictionary if sensible. + path: ./dictionaries/psdocs.txt + - name: pwshAliases + description: > + Dictionary of PowerShell aliases. Add entries to this dictionary for command and parameter + aliases to keep the main dictionary easier to use. + path: ./dictionaries/pwshAliases.txt + +# Defining patterns here makes it easier to understand the definitions for the ignore and include +# pattern lists (`*RegExpList`). Also allows us to document these patterns to some degree. +patterns: + - name: domain-azure-edge + description: Ignore misspellings caused by lowercase domain names for Azure edge domains. + pattern: /\S+\.azureedge\.net/ + - name: domain-windows-blob + description: Ignore misspellings caused by lowercase domain names Windows blob storage domains + pattern: /\S+\.blob\.core\.windows\.net/ + - name: domain-gallery + description: Ignore segments preceeding or following the powershellgallery domain name. + pattern: /(\S+\.)?powershellgallery\.com(\S+)?/ + - name: domains + description: Ignore apparent misspellings as components of well-known domain name. + pattern: + - domain-azure-edge + - domain-gallery + - domain-windows-blob + + - name: markdown-code-block-output + description: Ignore text in output code blocks. + pattern: '/(?:```[oO]utput[\s\S]*?```)/g' + - name: markdown-code-block-syntax + description: Ignore text in output code blocks. + pattern: '/(?:```[sS]yntax[\s\S]*?```)/g' + - name: markdown-code-blocks + description: Don't check spelling in output or syntax blocks. + pattern: + - markdown-code-block-output + - markdown-code-block-syntax + + - name: markdown-link-reference + description: Matches 'foobar' in '[foo bar][foobar]' + pattern: /(?<=\])\[[^\]]+\]/ + - name: markdown-link-inline + description: Matches '/foo/bar' in '[foo bar](/foo/bar)' + pattern: '/(?<=\])\([^\)]+\)/' + - name: markdown-link-definition + description: "Matches '/foo/bar' in '[foobar]: /foo/bar'" + pattern: '/(?<=\]:\s)(\s*((https?:)?|\/|\.{1,2}))(\/\S+)/' + - name: markdown-links + description: Don't check link definitions or references. + pattern: + - markdown-link-inline + - markdown-link-reference + - markdown-link-definition + + - name: registry-paths + description: Ignore Windows registry paths + pattern: /(HK(CR|CU|LM))(:\S*)?/ + + - name: wildcard-fragment-prefix + description: Ignore misspellings caused by partial words with a wildcard at the start. + pattern: '/[^\*]\*\w+/' + - name: wildcard-fragment-suffix + description: Ignore misspellings caused by partial words with a wildcard at the end. + pattern: '/\w+\*[^\*]/' + - name: wildcard-fragments + pattern: + - wildcard-fragment-prefix + - wildcard-fragment-suffix + +# Any patterns listed here are ignored for spellcheck. +# +# We ignore the URLs for inline markdown links, Markdown link references, and Markdown link +# reference definitions because these will otherwise be very noisy and they're not displayed to +# readers anyway. +# +# We ignore the spelling for all text in output code blocks for Markdown files because that text +# represents output from real commands and any spelling errors are not a fault in the documentation. +# +# We ignore registry paths, wildcard fragments, and components of well-known domains because those +# are intentionally or uncontrollably downcased or "incorrect" spellings. +ignoreRegExpList: + - domains + - markdown-code-blocks + - markdown-links + - registry-paths + - wildcard-fragments + +# The default locale for this documentation is US English. +language: 'en,en-US' + +# These settings are applied to combinations of language (file type) and locale. For any given file +# and locale, all matching dictionaries are applied. +languageSettings: + # Any file written in English + - languageId: '*' + locale: en + dictionaries: + - wordsEn + # Any file written in US English + - languageId: '*' + locale: en-US + dictionaries: + - wordsEn + # Any file written in British English + - languageId: '*' + locale: en-GB + dictionaries: + - wordsEnGb + # Any Markdown file + - languageId: markdown + locale: '*' + dictionaries: + - externalCommands + - fictionalCorps + - fileExtensions + - psdocs + - pwshAliases diff --git a/.vscode/cspell/psdocs/dictionaries/externalCommands.txt b/.vscode/cspell/psdocs/dictionaries/externalCommands.txt new file mode 100644 index 00000000..d2f95784 --- /dev/null +++ b/.vscode/cspell/psdocs/dictionaries/externalCommands.txt @@ -0,0 +1,52 @@ +AcroRd32 +Appinfo +appwiz +audiodg +Audiosrv +azurecli +ccmsetup +cd +certreq +chdir +conhost +csrss +distro +dpkg +elif +filesrv +ftype +gedit +iexplore +iisadmin +Intune +kubectl +LanmanServer +LanmanWorkstation +LSASRV +lsass +makecert +mkdir +mscorlib +msdtc +msseces +netcoreapp +Netlogon +netsh +netsvcs +Pandoc +rdpclip +RSAT +rstrui +setenv +svchost +SysmonLog +tlntsvr +w3wp +WFDBG +winget +Winlogon +Winmgmt +winver +Winword +WLIDSVC +xattr diff --git a/.vscode/cspell/psdocs/dictionaries/fictionalCorps.txt b/.vscode/cspell/psdocs/dictionaries/fictionalCorps.txt new file mode 100644 index 00000000..bf1523e0 --- /dev/null +++ b/.vscode/cspell/psdocs/dictionaries/fictionalCorps.txt @@ -0,0 +1,18 @@ +Adatum +Contoso +Fabrikam +Humongous +Lamna +Lucerne +Margie +Munson +Northwind +Proseware +Relecloud +Southridge +Tailspin +Tailwind +Trey +VanArsdel +Wingtip +Woodgrove diff --git a/.vscode/cspell/psdocs/dictionaries/fileExtensions.txt b/.vscode/cspell/psdocs/dictionaries/fileExtensions.txt new file mode 100644 index 00000000..487f945f --- /dev/null +++ b/.vscode/cspell/psdocs/dictionaries/fileExtensions.txt @@ -0,0 +1,10 @@ +adml +admx +asmx +bmil +cdxml +dylib +evtx +maml +mogg +msix diff --git a/.vscode/cspell/psdocs/dictionaries/psdocs.txt b/.vscode/cspell/psdocs/dictionaries/psdocs.txt new file mode 100644 index 00000000..2a1fbc98 --- /dev/null +++ b/.vscode/cspell/psdocs/dictionaries/psdocs.txt @@ -0,0 +1,279 @@ +ADSI +AMSI +Antimalware +assetid +Authenticode +autoclose +autocloses +autoclosing +autodetect +autogenerate +autogenerated +autogenerates +autoload +autoloaded +autoloads +autosave +autosaved +autosaves +backgrounded +backtick +backticks +bareword +barewords +bigendianunicode +blockquote +blockquotes +blog +bootstrapped +bootstrapper +bootstrapping +bootstraps +bytecode +callout +callouts +callstack +carmonm +ccontains +CentOS +cheatsheet +cheatsheets +checkboxes +cimv2 +clike +cmatch +cmdlet +cmdlets +cnotcontains +cnotlike +cnotmatch +codepage +codepages +computername +coreclr +CoreOS +CredSSP +creplace +customizations +DACL +datacenter +datacenters +Debian +deserialization +deserialize +deserialized +deserializes +devlang +DHCP +differentiators +discoverability +DISM +doc-a-thon +doc-a-thons +docfx +docset +documentationcenter +DPAPI +DWORD +endianness +Etag +Etags +eventlog +eventlogs +executables +façade +failover +finalizer +frontmatter +glob +globbing +globs +GUID +hardlink +hashtable +hashtables +hasthable +helpdesk +HKEY +hostname +hostnames +hotfix +hotfixes +HTTPS +icontains +ilike +imatch +inetpub +infographic +inotcontains +inotlike +inotmatch +IntelliSense +interpreted +intranet +intrinsics +ireplace +isnot +JavaScript +Kerberos +keypress +keypresses +Kubernetes +LASTEXITCODE +LCID +lifecycle +lockdown +markdig +msiexec +MSIL +MSRC +multibyte +multithreading +NETBIOS +netstandard +Newtonsoft +Newtonsoft's +NoBom +notcontains +notlike +notmatch +nslookup +NTFS +NTLM +nuget +nupkg +onboarding +openpublishing +openSUSE +pageable +parameterless +PATHEXT +pltfrm +POSIX +PowerShell +POWERSHELL_TELEMETRY_OPTOUT +preinstallation +prepopulate +prepopulated +prepopulates +prepopulating +prerelease +prereleases +psadapted +psbase +pscustomobject +PSES +psextended +PSHOME +PSHOST +PSISE +psobject +PSRP +PSSA +pstypenames +PSUICulture +PSWSMan +Punycode +pwsh +quickstartq +quickstarts +QWORD +Raspbian +Recurse +recurses +rehost +rehosting +rehosts +rehydrated +remoting +reparse +RHEL +RIPEMD160 +runbook +runbooks +runspace +runspaces +runtimes +SACL +SBOM +SBOMs +sbyte +scalability +scanability +SCCM +scriptable +scriptblocks +scripter +scripters +Sddl +SDII +SDKs +sdwheeler +sewhee +SIEM +SLES +Snapin +Snapins +Snover +SPACEBAR +Sqlcmd +SQLPS +stepover +struct +subcontainer +subcontainers +subexpression +subexpressions +subfolder +subfolder +subfolders +subkey +subkeys +subnet +subnets +subpipeline +subpipelines +subproperties +subproperty +symlink +systemdrive +taskbar +timespan +Titlecase +TMPDIR +triaged +Ubuntu +UMCI +undelimited +unencrypted +uninstallation +unjoin +unjoined +unjoining +unjoins +unlist +unlisted +unlisting +unlists +unlocalized +unmanaged +unregister +unregistered +unregistering +unregisters +untrusted +updateable +USERMODE +USERROLE +userspace +virtualized +virtualizes +walkthrough +WBEM +WDAC +webservice +WinCompat +wmicimv2 +workgroup +workgroups +wwwroot diff --git a/.vscode/cspell/psdocs/dictionaries/pwshAliases.txt b/.vscode/cspell/psdocs/dictionaries/pwshAliases.txt new file mode 100644 index 00000000..82bb34fd --- /dev/null +++ b/.vscode/cspell/psdocs/dictionaries/pwshAliases.txt @@ -0,0 +1,24 @@ +cd +cdd +chdir +clc +clhy +cli +clp +cls +clv +cnsn +cp +cpi +cpp +curi +cvpa +hfid +infa +ipmo +psedit +ruri +sasv +spps +spsv +usetx diff --git a/.vscode/extensions.json b/.vscode/extensions.json new file mode 100644 index 00000000..8ea76bc3 --- /dev/null +++ b/.vscode/extensions.json @@ -0,0 +1,12 @@ +{ + // See https://go.microsoft.com/fwlink/?LinkId=827846 + // for the documentation about the extensions.json format + "recommendations": [ + "docsmsft.docs-authoring-pack", + "marvhen.reflow-markdown", + "ms-vscode.powershell", + "shuworks.vscode-table-formatter", + "streetsidesoftware.code-spell-checker", + "wmaurer.change-case", + ] +} diff --git a/Docs/Becoming-a-contributor.md b/Docs/Becoming-a-contributor.md new file mode 100644 index 00000000..84e07fbe --- /dev/null +++ b/Docs/Becoming-a-contributor.md @@ -0,0 +1,50 @@ +# Becoming a contributor + +## Prerequisites + +Before your post can be published, you need to have a WordPress account. Use the following steps to +create an account. + +1. Open [https://devblogs.microsoft.com/powershell-community][02] +1. Click the Login button in the top-right corner of the page. +1. Login using one of the three options + + ![blog login][01] + +1. Fill in the following information in your profile: + - First and Last name + - The Display name should be your full name + - Add any social media links you wish to share (optional) + - Add a brief Bio explaining who you are / what your PowerShell experience is + - Add a profile picture (optional) + +By default, your account is added as a **Subscriber** in WordPress. We will review your profile +before you can be elevated to **Author** status. Your WordPress account must have **Author** +permission before your post can be published. + +## Getting started with GitHub and VS Code + +If you need help getting started with GitHub or the VS Code for markdown authoring, see the +_Set up and work locally_ section of the public +[Contributor's Guide][03]. + +This guide includes steps for the following items: + +1. [Creating a GitHub account][03] +1. [Installing Git, VS Code, and the Docs Authoring Pack][05] +1. [Setting up your local Git repository][04] +1. [Git and GitHub fundamentals][06] +1. [An explanation of the full GitHub workflow][07] + +We also recommend installing the [posh-git][08] module from the PowerShell Gallery. This module makes +it easier to use Git from the PowerShell command line. + + +[01]: ./media/Becoming-a-contributor/blog-login.png +[02]: https://devblogs.microsoft.com/powershell-community +[03]: https://docs.microsoft.com/contribute/get-started-setup-github +[04]: https://docs.microsoft.com/contribute/get-started-setup-local +[05]: https://docs.microsoft.com/contribute/get-started-setup-tools +[06]: https://docs.microsoft.com/contribute/git-github-fundamentals +[07]: https://docs.microsoft.com/contribute/how-to-write-workflows-major +[08]: https://www.powershellgallery.com/packages/posh-git diff --git a/Docs/Creating-a-new-post.md b/Docs/Creating-a-new-post.md new file mode 100644 index 00000000..ed8e237f --- /dev/null +++ b/Docs/Creating-a-new-post.md @@ -0,0 +1,83 @@ +# Creating a new post + +1. Always create a _working branch_ in your local repo before starting a new article. Avoid working + in the `main` branch. +1. Create a new `.md` file in `Posts/YYYY/MM` directory. For example, posts scheduled to be + published in February of 2021 go in the `Posts/2021/02` folder. Create the monthly folder if it + doesn't exist yet. + - Filenames should only use the following characters: A-Z (upper and lower), 0-9, and hyphen (`-`) + - Don't use spaces or special characters in filenames + - Separate words in the filename with hyphens + - The filename must include the `.md` file extension +1. Write the blog post! + - Use [GitHub flavored markdown][1]. + - The blog post **MUST** have this header: + + ```yaml + --- + post_title: 'Post Title' + user_login: The author's Word Press username, not GitHub ID + author1: + author2: The author's Word ress username, not GitHub ID + author3: The author's Word Press username, not GitHub ID + post_slug: + categories: existingcategory1, existingcategory2 + tags: tag1, tag2 + summary: summary of the post + --- + + Add your blog post here + ``` + + - The `post_title`, `summary`, and `user_login` are required fields. + - `post_slug` is an identifier for your post and it becomes the end portion of the URL for the + post. + - If the slug exists in Word Press, the post matching the slug is updated. + - If the slug does not exist in Word Press, a new post is created. + - If you don't provide a slug, Word Press creates a slug when it creates the draft. + - Use lowercase letters, numbers, and hyphens. Use hyphens to separate words rather than other + punctuation. + - For more information about slugs, see + [What is a Word Press slug?](https://www.wpkube.com/wordpress-slug/) + - `categories` - one or more category strings separated by commas + - The category values must already exist in your blog + - `tags` - one or more strings separated by commas + - New values are added as available tags for your blog in Word Press + - `summary` - This is the short description of the post that shows in listing of posts on the + main page of your blog + - `user_login` or `author1` - can be used to add a single author + - `author2` or `author3` - should be used when adding up to two additional authors + + - PowerShell code snippet: + + ~~~markdown + ```powershell + Get-Alias dir # this will be highlighted with PowerShell syntax + ``` + ~~~ + + - Console output snippet: + + ~~~markdown + ```powershell-console + CommandType Name Version Source + ----------- ---- ------- ------ + Alias dir -> Get-ChildItem + ``` + ~~~ + +1. Read and follow the rules in the [Reviewer's Guide][2]. Edit your post based on these rules + before submitting the PR. This saves the reviewers a lot of time and your post can be approved + more quickly. + +## Publishing draft to blog + +After submitting your Pull Request, the blog admins will review the post. They may suggest editorial +changes to improve grammar and readability. They may also require specific changes before we can +publish. Once the pull request is merged, the post is automatically copied to Word Press as a draft. +From there, the Blog admins will verify that the post renders correctly, make any formatting changes +required, and publish the post. + + +[1]: ./Markdown-cheatsheet.md +[2]: ./Reviewers-Guide.md diff --git a/Docs/GitHub-workflow-for-new-post.md b/Docs/GitHub-workflow-for-new-post.md new file mode 100644 index 00000000..58bbffb3 --- /dev/null +++ b/Docs/GitHub-workflow-for-new-post.md @@ -0,0 +1,25 @@ +# GitHub workflow for a new post + +The following image illustrates the workflow for using Git and GitHub to create a new post for the +Community blog. The steps shown in red are a one-time action and are covered in Setup GitHub for +[local workflow][1]. The numbered steps (in black) are described in the table below. + +![Blog GitHub workflow][2] + +| Steps | Description of steps | Git command / GitHub actions | | +| ----- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --- | +| 0 | Checkout the main branch | `git checkout main` | | +| 1 | Sync the main branch | `git pull upstream main`
`git push origin main` | | +| 2 | Create a new working branch | git checkout -b new-post-name | | +| 3 | Create new content | Use VS Code to create new blog post | | +| 4-5 | Commit changes to local repo | `git add -A`
`git commit -m 'commit message'` | | +| 6 | Push working branch to fork | git push origin new-post-name | | +| 7 | Submit pull request | Go to `https://github.com//Community-Blog/pulls` and click the **New pull request** button.

`Base repository: PowerShell/Community-Blog base: main <-- head repository: /Community-Blog compare: new-post-name`

Fill out the pull request description and click **Submit**. | | +| 8 | PR is reviewed | Make the necessary changes based on the review feedback. | | +| 9 | PR is merged | Go to step 10 | | +| 10 | Cleanup unneeded branch info | `git checkout main`
`git push origin --delete new-post-name`
`git branch -D new-post-name`

The `git push` command deletes the branch in your fork and deletes the tracking branch from your local repo. The `git branch` command delete the branch from your local repo. | | +| 11 | Start new post | Go to step 0 | . | + + +[1]: ./Setup-GitHub-for-Local-Workflow.md +[2]: ./media/GitHub-workflow-for-new-post/Blog-gitflow.png diff --git a/Docs/Ideas-for-Posts.md b/Docs/Ideas-for-Posts.md new file mode 100644 index 00000000..45e9f090 --- /dev/null +++ b/Docs/Ideas-for-Posts.md @@ -0,0 +1,53 @@ +# Ideas for a new post + +## Share your experiences + +Ideas for posts can come from anywhere, but the best posts come from your own experience. Think +about a time when you solved a unique problem using PowerShell or learned something new about +PowerShell. Personal stories make good content. + +Blog posts should be your own original content. It's acceptable to reuse content you posted to +another blog as long as you own the rights to reuse that content. + +## Update a post from the Scripting Blog + +There is a lot of useful content on the old [Scripting Blog][1], but the information may be +incomplete, no longer accurate, or needs to be updated for the current version of PowerShell. + +If you find a post on the old blog that you think could be updated, create a new issue using the +[Request update to old post from The Scripting Blog][2] issue template. Include the following +information: + +- Link to the post in the old blog +- Description of what needs to be changed + +Then you can start working on a rewrite or you wait for someone else to create an update. + +## Look for issues labeled `up-for-grabs` + +Anyone can submit ideas for new posts or requests for updates to existing posts, but not everyone is +ready to write that post. Issues that are labeled `up-for-grabs` are open to anyone who might be +interested in writing that post. You can get a complete list of these issues by +[filtering the view][3] on that label. + +## Acceptable content ideas + +The intended purpose of this blog is to provide a blogging platform for the PowerShell Community, +both internal and external to Microsoft's PowerShell team. Posts to the blog can discuss products +and technologies that aren't part of the core PowerShell product or even made by Microsoft, as long +as the content is relevant to PowerShell users and isn't marketing those products. + +Acceptance of any blog post is done at the sole discretion of the Blog admins. + +Acceptable posts meet one or more of the following criteria: + +- Show how to use PowerShell to solve a specific problem or scenario +- Explain PowerShell usage in more detail than provided in the documentation +- Doesn't contain marketing materials or product announcements + - Examples using non-Microsoft products and modules are allowed as long as the purpose is to + demonstrate a solution + + +[1]: https://devblogs.microsoft.com/scripting +[2]: https://github.com/PowerShell/Community-Blog/issues/new?assignees=&labels=post-update&template=Request_Update.md&title=Update+request +[3]: https://github.com/PowerShell/Community-Blog/issues?q=is%3Aissue+is%3Aopen+label%3Aup-for-grabs diff --git a/Docs/Markdown-cheatsheet.md b/Docs/Markdown-cheatsheet.md new file mode 100644 index 00000000..e9db45ba --- /dev/null +++ b/Docs/Markdown-cheatsheet.md @@ -0,0 +1,248 @@ +# Markdown cheatsheet + +## Frontmatter + +All posts must have the following YAML blog at the top of the markdown file. + +```yaml +--- +post_title: 'Post Title' +username: Author username as seen in wordpress, not GitHub ID +categories: existingcategory1, existingcategory2 +tags: tag1,tag2 +featured_image: check instructions below +summary: summary of the post +--- +``` + +Fill in all sections of the header. The `post_title`, `summary`, and `username` are required +fields. + +For featured image, please follow the guidance for images below. + +## General formatting + +### Headers (ATX Style) + +Use `#` in front of a header to identify it as a header and also to auto-generate a section-link for +that header. This + +```markdown +# for H1 +## for H2 +### for H3 +#### for H4 +##### for H5 +###### for H6 +``` + +While Markdown supports other formats for headers, please use only ATX Style headers. + +### Emphasis + +- Italics - use underscores `_` (use without spaces) - _this is Italics_ +- Bold - use double asterisks `**` (use without spaces) - **this is bold** +- Strikethrough - use two tildes (use without spaces) - ~~Scratch this~~ + +### Lists + +~~~markdown +1. First ordered list item +1. Another item +1. Actual numbers don't matter. Use 1. for every item. This makes it easier to reorder + - Unordered sub-list. + - Use hyphens - for unordered list item. Markdown supports other characters but the asterisk is + too easily confused as emphasis. Using the hyphen avoids this confusion. +1. Ordered sub-list + 1. First subitem + + You can have properly indented paragraphs within list items. Notice the blank line above, and + the leading spaces. The first character of the paragraph should line up with the first + character of the list item. +~~~ + +1. First ordered list item +1. Another item +1. Actual numbers don't matter. Use `1.` for every item. This makes it easier to reorder + - Unordered sub-list. + - Use hyphens `-` for unordered list item. Markdown supports other characters but the asterisk is + too easily confused as emphasis. Using the hyphen avoids this confusion. +1. Ordered sub-list + 1. First subitem + + You can have properly indented paragraphs within list items. Notice the blank line above, and + the leading spaces. The first character of the paragraph should line up with the first + character of the list item. + +### Code blocks + +Code blocks can be added using the triple-backtick ` ``` ` block style. See example below. + +~~~markdown +```powershell +Invoke-RestMethod +``` +~~~ + +```powershell +Invoke-RestMethod +``` + +If you are mixing code with output, use the `powershell-console` language label for the code block. +For example: + +~~~markdown +```powershell-console +PS C:\> # Get the current date +PS C:\> Get-Date +08 January 2021 11:24:46 + +# Store the date in a variable +$Now = Get-Date +$Now +08 January 2021 11:24:47 +``` +~~~ + +### Code within text + +Code within a paragraph can be added using single-backticks. See example below. + +```markdown +This is a sentence with `inline code` in between. +``` + +This is a sentence with `inline code` in between. + +### Images + +- Images in public space e.g. public GitHub repo + + If the images are in a public space like docs, or already in the blog media folder, or a public + GitHub repo, simply add them in the standard markdown format as shown below. Remember to add alt + text for all your images. + + ![alttext][1] + + If you want images from public site, to be copied over to your blog's media folder, then follow + the steps mentioned in the images in private repo section. + +- Images in the GitHub repo + + To include images in your post you must: + + 1. Create a `media/` folder under the current month's folders. `` + should match the name of your markdown file (without the file extension). + 1. Put all images for the post in that folder. + 1. Link to the image using the standard markdown syntax: + + ```markdown + ![alt-text](./media//image-name.ext) + ``` + +### Videos + +For videos directly uploaded to the WordPress media folder, you can add the video links in GitHub +with this `video` shortcode. + +```markdown +[video src="https://devblogs.microsoft.com/powershell/wp-content/uploads/sites/30/2020/05/PSNativePSPathResolution.mp4"] +``` + +For videos uploaded to the GitHub repo, if you add a link to the video in GitHub repo, we don't yet +have a way to bring it into the media folder in WordPress. So all such videos need to be uploaded to +the WordPress media library then added to the draft in WordPress, or added to the draft in GitHub +via the video shortcode, example shown above. + +#### YouTube Videos + +You can add YouTube videos. The typical YouTube embed code looks like this : + +```html + +``` + +You can use the `iframe` shortcode for simpler syntax. To align the video use `

` tag as shown +in this example: + +```html +

+[iframe src="https://www.youtube.com/embed/hLFyycJVo0I" width="320" height="240"] +

+``` + +### Links + +Links can be added using standard markdown link format. See example below. + +```markdown +[tidy up the ASP.NET Core shared framework](https://blogs.msdn.microsoft.com/webdev/2018/10/29/a-first-look-at-changes-coming-in-asp-net-core-3-0/), +Json.NET is being removed from the shared framework and now needs to be added as a package. +``` + +### Tables + +Markdown supports tables with alignment (left,center,right). See examples below. + +```markdown +| Syntax | Description | Test Text | +| :-------- | :---------: | ----------: | +| Header | Title | Here's this | +| Paragraph | Text | And more | +``` + +This is what the basic table will look like: + +| Syntax | Description | Test Text | +| :-------- | :---------: | ----------: | +| Header | Title | Here's this | +| Paragraph | Text | And more | + +You can use HTML tags if you need more attributes for a table. + +## Blog-specific formatting + +### Embedding media files + +The blogging platform supports embedded media (videos, audio, podcasts, etc.) in your posts. You can +add embedded media using the WordPress editor after the PR has been merged. Or you can use special +markup in your Markdown source in GitHub. If you want to do this, add a comment to your PR +requesting assistance from a Blog admin to create the proper link. + +### Alert boxes + +Similar to the Docs platform, our WordPress platform supports Alert boxes used for calling out +important information. You can use the following syntax in your Markdown post to create an alert. + +```markdown +[alert type="note" heading="Note"] +Information the user should notice even if skimming. +[/alert] + +[alert type="tip" heading="Tip"] +Optional information to help a user be more successful. +[/alert] + +[alert type="important" heading="Important"] +Essential information required for user success. +[/alert] + +[alert type="caution" heading="Caution"] +Negative potential consequences of an action. +[/alert] + +[alert type="warning" heading="Warning"] +Dangerous certain consequences of an action. +[/alert] +``` + +You can also customize the heading as appropriate. The following image shows what each alert type +looks like (with and without a heading). + +![Alerts][2] + + +[1]: https://devblogsarchiv.wpengine.com/wp-content/uploads/2020/02/allmycomments.jpg +[2]: ./media/Markdown-cheetsheet/alerts.png \ No newline at end of file diff --git a/Docs/README.md b/Docs/README.md new file mode 100644 index 00000000..d895e794 --- /dev/null +++ b/Docs/README.md @@ -0,0 +1,54 @@ +# Welcome to the Community-Blog contributor guide + +The Wiki contains documentation on how to create a new blog post and how we operate this blog. + +Participation in this blog is governed by the [Microsoft Open Source Code of Conduct][1] or the +[.NET Foundation Code of Conduct][2]. For more information, see the [Code of Conduct FAQ][3]. + +Making contributions to the blog is very similar to making contributions to docs.microsoft.com. To +get started, you require the following things: + +1. A WordPress account (see [Becoming a contributor][4]) +1. A GitHub account +1. Familiarity with Markdown and VS Code +1. Familiarity with the GitHub workflow + +If you need help getting started with GitHub or the Docs authoring process, see the _Set up and work +locally_ section of the public [Contributor's Guide][5]. + +Formatting of the content should follow the same rules published in the +[PowerShell-specific Contributors Guide][6]. and the [PowerShell Style Guide][7]. + +## Table of Contents + +- [1 - Becoming a contributor][8] +- [2 - Creating a new-post][9] +- [3 - Submitting a PR][10] + +### Appendices + +- [A - Markdown cheatsheet][11] +- [B - Setup GitHub for local workflow][12] +- [C - GitHub workflow for a new post][13] +- [D - Ideas for Posts][14] + +## Operations Guide + +- [Reviewer's Guide][15] + + +[1]: https://opensource.microsoft.com/codeofconduct/ +[2]: https://dotnetfoundation.org/code-of-conduct +[3]: https://opensource.microsoft.com/codeofconduct/faq/ +[4]: Becoming-a-contributor +[5]: https://docs.microsoft.com/contribute/get-started-setup-github +[6]: https://docs.microsoft.com/powershell/scripting/community/contributing/overview +[7]: https://docs.microsoft.com/powershell/scripting/community/contributing/powershell-style-guide +[8]: Becoming-a-contributor.md +[9]: Creating-a-new-post.md +[10]: Submitting-a-PR.md +[11]: Markdown-cheatsheet.md +[12]: Setup-GitHub-for-Local-Workflow.md +[13]: GitHub-workflow-for-new-post.md +[14]: Ideas-for-Posts.md +[15]: Reviewers-Guide.md diff --git a/Docs/Reviewers-Guide.md b/Docs/Reviewers-Guide.md new file mode 100644 index 00000000..aff99d69 --- /dev/null +++ b/Docs/Reviewers-Guide.md @@ -0,0 +1,110 @@ +# Reviewer's Guide + +This is a summary of rules to apply when writing new or updating existing articles. See other +articles in the Contributor's Guide for detailed explanations and examples of these rules. + +## PR Quality + +- Submitted from a working branch (not from main) +- PR contains only one post +- The post is in the correct folder with a correctly structured filename + `Posts/yyyy/mm/simple-title-of-post.md` + - No spaces in filenames + - Lowercase preferred + - Must have `.md` file extension + +## User profile + +- The submitter has a valid WordPress profile with appropriate personal information + - Full name + - Any links included are valid and appropriate + - The picture is appropriate (avatars are OK - prefer actual photo) +- The submitter has been changed to the **Author** role in WordPress + +## Content quality + +- Reasonable grammar and spelling +- Correct spelling and usage of brands (eg. "PowerShell" not "Powershell") +- Content is designed to teach or inform not market or sell +- Submitter has proper rights to the content they are submitting + +## Metadata + +All blog posts must include the YAML frontmatter: + +```yaml +--- +post_title: +username: <Author username as seen in wordpress, not github ID> +categories: <choose from list of predefined categories> +tags: <choose from list of predefined tags> +featured_image: <Optional Image url> +summary: <Summary of the post - short one-line description> +--- +``` + +## Formatting + +- Backtick syntax elements that appear, inline, within a paragraph + - Cmdlet names `Verb-Noun` + - Variable `$counter` + - Syntactic examples `Verb-Noun -Parameter` + - File paths `C:\Program Files\PowerShell`, `/usr/bin/pwsh` + - URLs that aren't meant to be clickable in the document + - Property or parameter values +- Use bold for property names, parameter names, class names, module names, entity names, object or + type names + - Bold is used for semantic markup, not emphasis + - Bold - use asterisks `**` +- Italic - use underscore `_` + - Only used for emphasis, not for semantic markup +- Line breaks at 100 columns - helps when reviewing diffs + - Use the [Reflow Markdown][1] extension in VS Code to help +- No hard tabs - use spaces only +- No trailing spaces on lines + +### Headers + +- DO NOT use the H1 header - WordPress automatically puts the title at the top of the post +- Use [ATX Headers][2] only +- Use sentence case for all headers +- Don't skip levels - no H3 without an H2 +- Max depth of H3 or H4 +- Blank line before and after + +### Code blocks + +- Blank line before and after the code block (not inside the code block) +- Use tagged code fences - `powershell`, `powershell-console`, or other appropriate language tags +- Untagged fence - syntax blocks or other shells +- Put output in separate `powershell-console` code block +- Don't use the PowerShell prompt in code blocks unless: + - You are showing an example meant to be used on the command line. + - Use `PS>` for the prompt unless the path is important to the example. + +### Lists + +- Blank line before first item and after last item +- Indented properly + - Additional lines for an item should line up with first character after the list marker +- Bullet - use hyphen (`-`) not asterisk (`*`) - too easy to confuse with emphasis +- For numbered lists, all numbers are "1." + +## Terminology + +- PowerShell vs. Windows PowerShell vs. PowerShell Core +- See [Product Terminology][3] + +## Linking to other websites + +- Do not include locales in URLs linking to Microsoft properties (eg. remove `/en-us` from URL) +- Do not include the `?view=<version>` query parameter when linking to docs.microsoft.com +- All URLs to websites should use HTTPS unless that is not valid for the target site +- Image links should have unique alt text +- No bare URLs - Use standard markdown link syntax `[text of link](https://site.domain/path/to/page#anchor)` +- The link text should be the title of the page or the anchor that you link to + +<!-- link references --> +[1]: https://marketplace.visualstudio.com/items?itemName=marvhen.reflow-markdown +[2]: https://github.github.com/gfm/#atx-headings +[3]: https://learn.microsoft.com/powershell/scripting/community/contributing/product-terminology diff --git a/Docs/Setup-GitHub-for-Local-Workflow.md b/Docs/Setup-GitHub-for-Local-Workflow.md new file mode 100644 index 00000000..fb07a5b6 --- /dev/null +++ b/Docs/Setup-GitHub-for-Local-Workflow.md @@ -0,0 +1,143 @@ +# Setting up Git/GitHub for working locally + +To contribute to the blog, you can make and edit Markdown files locally by cloning the corresponding +**PowerShell/Community-Blog** repository. Microsoft requires you to fork the repository into your +own GitHub account so that you have read/write permissions there to store your proposed changes. +Then you use pull requests to merge changes into the read-only central shared repository. + +![GitHub Triangle][1] + +## Fork the repository + +Create a fork of the **PowerShell/Community-Blog** repository into your own GitHub account using +the GitHub website. + +A personal fork is required since the main repositories provide read-only access. To make changes, +you must submit a [pull request][2] from your fork into the main repository. To facilitate this +process, you first need your own copy of the repository. A GitHub _fork_ serves that purpose. + +1. Go to [https://github.com/PowerShell/Community-Blog][3] + and click the **Fork** button on the upper right. + + ![GitHub profile example][4] + +2. If you are prompted, select your GitHub account tile as the destination where the fork should be + created. This prompt creates a copy of the repository within your GitHub account, known as a + fork. + +## Choose a local folder + +Make a local folder to hold a copy of the repository locally. Some repositories can be large; up to +5 GB for Community-Blog for example. Choose a location with available disk space. + +1. Choose a foldername should be easy for you to remember and type. For example, consider a root + folder `C:\Git\` or make a folder in your user profile directory `~/Documents/Git/` + + > [!IMPORTANT] + > Avoid choosing a local folder path that's nested inside of another git repository folder + > location. While it's acceptable to store the git cloned folders adjacent to each other, nesting + > git folders inside one another causes errors for the file tracking. + +1. From the PowerShell command line, change directory (`cd`) into the folder that you created for + hosting the repository locally. Note that Git Bash uses the Linux convention of forward-slashes + instead of back-slashes for folder paths. + + For example, `cd ~/Documents/Git/` + +## Create a local clone + +Prepare to run the **clone** command to pull a copy of a repository (your fork) down to your device +on the current directory. + +### Authenticate using Git Credential Manager + +If you installed the latest version of Git for Windows and accepted the default installation, Git +Credential Manager is enabled by default. Git Credential Manager makes authentication much easier +because you don't need to recall your personal access token when re-establishing authenticated +connections and remotes with GitHub. + +1. Run the **clone** command, by providing the repository name. Cloning downloads (clone) the forked + repository on your local computer. + + > [!Tip] + > You can get your fork's GitHub URL for the clone command from the **Clone or download** button + > in the GitHub UI: + > + > ![Clone or download][5] + + Be sure to specify the path to *your fork* during the cloning process, not the main repository + from which you created the fork. Otherwise, you cannot contribute changes. Your fork is + referenced through your personal GitHub user account, such as + `github.com/<github-username>/<repo>`. + + ```powershell + git clone https://github.com/<github-username>/<repo>.git + ``` + + Your clone command should look similar to this example: + + ```powershell + git clone https://github.com/MyGitAccount/Community-Blog.git + ``` + +1. When you're prompted, enter your GitHub credentials. + + ![GitHub Login][6] + +1. When you're prompted, enter your two-factor authentication code. + + ![GitHub two-factor authentication][7] + + > [!NOTE] + > Your credentials are saved and used to authenticate future GitHub requests. You only need to + > do this authentication once per computer. + +1. The clone command downloads a copy of the files from your fork of the repository into a new + folder on the local disk. The new folder is created within the current folder. It may take a few + minutes, depending on the repository size. You can explore the folder to see the structure once + it is finished. + +## Configure remote upstream + +After cloning the repository, set up a read-only remote connection to the main repository named +**upstream**. You use the upstream URL to keep your local repository in sync with the latest changes +made by others. The **git remote** command is used to set the configuration value. You use the +**fetch** command to refresh the branch info from the upstream repository. + +1. Use the following commands. + + ```powershell + cd Community-Blog + git remote add upstream https://github.com/PowerShell/Community-Blog.git + git fetch upstream + ``` + +1. View the configured values and confirm the URLs are correct. Ensure the **origin** URLs point to + your personal fork. Ensure the **upstream** URLs point to the main repository, such as + MicrosoftDocs or Azure. + + ```powershell + git remote -v + ``` + + Example remote output is shown. A fictitious git account named MyGitAccount is configured with a + personal access token to access the repo Community-Blog: + + ```output + origin https://github.com/MyGitAccount/Community-Blog.git (fetch) + origin https://github.com/MyGitAccount/Community-Blog.git(push) + upstream https://github.com/PowerShell/Community-Blog.git (fetch) + upstream https://github.com/PowerShell/Community-Blog.git (push) + ``` + +1. If you made a mistake, you can remove the remote value. To remove the upstream value, run the + command `git remote remove upstream`. + +<!-- link references --> +[1]: ./media/Setup-GitHub-for-Local-Workflow/git-and-github-initial-setup.png +[2]: https://docs.microsoft.com/contribute/git-github-fundamentals +[3]: https://github.com/PowerShell/Community-Blog +[4]: ./media/Setup-GitHub-for-Local-Workflow/fork.png +[5]: ./media/Setup-GitHub-for-Local-Workflow/clone-or-download.png +[6]: ./media/Setup-GitHub-for-Local-Workflow/github-login.png +[7]: ./media/Setup-GitHub-for-Local-Workflow/github-2fa.png diff --git a/Docs/Submitting-a-PR.md b/Docs/Submitting-a-PR.md new file mode 100644 index 00000000..6574c491 --- /dev/null +++ b/Docs/Submitting-a-PR.md @@ -0,0 +1,46 @@ +# Submitting a Pull Request (PR) + +1. Create your a new MD file for your post + + - Always create a _working branch_ in your local repo before starting a new article. Avoid + working in the `main` branch. + - Create the new post in the folder for the current month (E.g. `Posts/2021/02` for February of + 2021). + - All images you want to include go in the `media` folder of the current month. + - If the media folder doesn't exist, create it. + - Create a subfolder that matches the name of you post's MD file in the `media` folder. + +1. Write your content in markdown. + + - Be sure to include the YAML frontmatter in your file. + - Follow the guidance in our [Reviewer's Guide][1]. + - You don't need to repeat the title as an H1 header. The first header in your post should be + H2. This header DOES NOT need to be the first lin of you post after the frontmatter. + +1. Push your _working branch_ to your fork in GitHub. +1. Create a PR to merge the _working branch_ of your fork into the `main` branch of the + **PowerShell/Community-Blog** repository. +1. Fill out the PR template and click submit. + + - Include specific instructions in the template if you want the post to be published at a + specific date and time. + +## After submitting your PR + +### Contributor License Agreement + +If you are contributing for the first time, you will be asked to complete a short Contribution +License Agreement (CLA). After the CLA step is cleared, your pull request is processed. + +### PR Review process + +Your PR will be reviewed by the Blog staff. They may have editorial suggestions. Please fix issues +and accept the editorial changes. + +Once your PR has been reviewed and approved, it will be merged. Once merged the post is +automatically copied to WordPress where the WP admins will preview the rendering. They may have to +make minor changes to fix issues created by the translation to WordPress. If the Preview is good, +the post will be published. + +<!-- link references --> +[1]: https://github.com/PowerShell/Community-Blog/wiki/Reviewers-Guide diff --git a/Docs/media/Becoming-a-contributor/blog-login.png b/Docs/media/Becoming-a-contributor/blog-login.png new file mode 100644 index 00000000..8dfd642f Binary files /dev/null and b/Docs/media/Becoming-a-contributor/blog-login.png differ diff --git a/Docs/media/GitHub-workflow-for-new-post/Blog-gitflow.png b/Docs/media/GitHub-workflow-for-new-post/Blog-gitflow.png new file mode 100644 index 00000000..079d9bf5 Binary files /dev/null and b/Docs/media/GitHub-workflow-for-new-post/Blog-gitflow.png differ diff --git a/Docs/media/Markdown-cheetsheet/alerts.png b/Docs/media/Markdown-cheetsheet/alerts.png new file mode 100644 index 00000000..aa51a876 Binary files /dev/null and b/Docs/media/Markdown-cheetsheet/alerts.png differ diff --git a/Docs/media/Setup-GitHub-for-Local-Workflow/clone-or-download.png b/Docs/media/Setup-GitHub-for-Local-Workflow/clone-or-download.png new file mode 100644 index 00000000..e6c0b207 Binary files /dev/null and b/Docs/media/Setup-GitHub-for-Local-Workflow/clone-or-download.png differ diff --git a/Docs/media/Setup-GitHub-for-Local-Workflow/edit-article.png b/Docs/media/Setup-GitHub-for-Local-Workflow/edit-article.png new file mode 100644 index 00000000..d64c1bc8 Binary files /dev/null and b/Docs/media/Setup-GitHub-for-Local-Workflow/edit-article.png differ diff --git a/Docs/media/Setup-GitHub-for-Local-Workflow/fork.png b/Docs/media/Setup-GitHub-for-Local-Workflow/fork.png new file mode 100644 index 00000000..712799b8 Binary files /dev/null and b/Docs/media/Setup-GitHub-for-Local-Workflow/fork.png differ diff --git a/Docs/media/Setup-GitHub-for-Local-Workflow/git-and-github-initial-setup.png b/Docs/media/Setup-GitHub-for-Local-Workflow/git-and-github-initial-setup.png new file mode 100644 index 00000000..c1d3096f Binary files /dev/null and b/Docs/media/Setup-GitHub-for-Local-Workflow/git-and-github-initial-setup.png differ diff --git a/Docs/media/Setup-GitHub-for-Local-Workflow/gitbash-start.png b/Docs/media/Setup-GitHub-for-Local-Workflow/gitbash-start.png new file mode 100644 index 00000000..be8c53e3 Binary files /dev/null and b/Docs/media/Setup-GitHub-for-Local-Workflow/gitbash-start.png differ diff --git a/Docs/media/Setup-GitHub-for-Local-Workflow/github-2fa.png b/Docs/media/Setup-GitHub-for-Local-Workflow/github-2fa.png new file mode 100644 index 00000000..32558c86 Binary files /dev/null and b/Docs/media/Setup-GitHub-for-Local-Workflow/github-2fa.png differ diff --git a/Docs/media/Setup-GitHub-for-Local-Workflow/github-login.png b/Docs/media/Setup-GitHub-for-Local-Workflow/github-login.png new file mode 100644 index 00000000..3de40bd9 Binary files /dev/null and b/Docs/media/Setup-GitHub-for-Local-Workflow/github-login.png differ diff --git a/Posts/2021/03/Caps-lock-ToUpper.md b/Posts/2021/03/Caps-lock-ToUpper.md new file mode 100644 index 00000000..356619d8 --- /dev/null +++ b/Posts/2021/03/Caps-lock-ToUpper.md @@ -0,0 +1,206 @@ +--- +post_title: Can I Enable the Caps Lock Key? +username: tfl@psp.co.uk +Catagories: PowerShell +tags: Caps Lock, string, ToUpper() +Summary: How to enable all upper case input +--- + +**Q:** I have a script where users enter some information. +This information needs to be entered in all capital letters, so my instructions say, “Please make sure the Caps Lock key is on before entering the information.” +They don’t always do that, however. +Is there a way to turn the Caps Lock key on and off using a script? + +**A:** I don't know how to run the key off and on, but with PowerShell, there is a way to mimic the effect of turning on the Caps Lock key. + +## User Input Considered Harmful + +Let's start with the observation that all user input is harmful. +One of my earliest IT heroes, Edsger Dijkstra, published a seminal letter [Go To Statement Considered Harmful](https://homepages.cwi.nl/~storm/teaching/reader/Dijkstra68.pdf) in 1968 which began the structured programming revolution. +And this is one reason, by the way, why PowerShell has no goto statement. +The phrase "Considered Harmful" is also a well-known phrase that has a Wikipedia entry at [Considered Harmful](https://wikipedia.org/wiki/Considered_harmful#:~:text=Considered%20harmful%20was%20popularized%20among,the%20day%20and%20advocated%20structured). +In general, I consider all user input potentially harmful, capable of doing damage until and unless you thoroughly validate it first. + +## Is User Input Really Harmful? + +I was a verification programmer at university and got paid an hourly rate plus a bonus for finding bugs. +I made far more than my hourly wage by simply testing conditions outside what the developers considered "normal". +In other words potentially harmful. + +If an instruction said, "Enter a number between 1 and 6", I tried -124, 0, 7, 42, 999999, and so on. +This approach inevitably led to several bugs (and several nice bug bounties). +Ever since then, I have always taught my students never ever to accept user input unchecked. +And that includes having all upper case input if that is what your application requires. + +Another example of unchecked user input being harmful is SQL injection attacks. +You can read more about these attacks and how you can prevent it at [What is SQL Injection (SQLi) and How to Prevent It](https://www.acunetix.com/websitesecurity/sql-injection/) + +So, in general, you should never trust any user input without validating it first. +Although you ask the user to type her name in all upper case, I'll bet that many just won't. + +So what does the Caps Lock actually do? +When you type characters into a form or a console, you might type them like this: + +```powershell-console +this is my sentence. +``` + +If you switch on the Caps Lock key, the operating system and your hardware makes those characters appear like this: + +```console +THIS IS MY SENTENCE. +``` + +So how can we achieve that same effect in a script? +Simple: we accept the input as the user typed it. +Then we make sure it's all upper case before using it. + +Let's start with getting the user input in the first place. + +## Getting User Input + +There are several ways to get user input from within a script. +A common approach with PowerShell scripts is to use the `Read-Host` command. +This cmdlet reads a line of input from the console and returns it to the script as a string. +For more information on this cmdlet, see the [Read-Host help file](https://docs.microsoft.com/powershell/module/microsoft.powershell.utility/read-host). + +There are other ways to get user input, such as using [Windows Forms](https://docs.microsoft.com/powershell/scripting/samples/creating-a-custom-input-box) or WPF. +You might even use a legacy [Visual Basic `Inputbox`](https://docs.microsoft.com/dotnet/api/microsoft.visualbasic.interaction.inputbox). +But with each of these methods, you still have the underlying issue of making sure the string the user enters is all upper-case before you use it further. + +Suppose you wanted to ask the user for their name (and you really need it to be upper case). +You could ask for, accept, and then display user input like this: + +```powershell-console +PS C:\Foo> $Answer = Read-Host -Prompt 'Please Enter Your Name In ALL Upper case' +Please Enter Your Name In ALL Upper case: Thomas Lee +PS C:\Foo> $Answer +Thomas Lee +``` + +But that is not in upper case, I hear you say. +Yes, true - but there is just one more step. +Be patient, grasshopper. + +## Converting a String to Upper Case + +As I mentioned, when you use `Read-Host`, PowerShell returns the input to you as a string. +Even if you enter a number (say 42) PowerShell still treats this as a string containing two characters, like this: + +```powershell-console +PS C:\> $Answer = Read-Host -Prompt "Please Enter Your Name In ALL Upper case" +Please Enter Your Name In ALL Upper case: 42 +PS C:\> $Answer.GetType().FullName +System.String +``` + +This matters because the `System.String` .NET class has a very useful method, `ToUpper()`. +The `ToUpper()` method converts the string to all upper case and returns a new, all upper case, string. +So to convert the string you entered and stored in `$Answer`, you use the `ToUpper()` method like this: + +```powershell-console +PS C:\> $Answer = Read-Host -Prompt 'Enter Your Name In ALL Upper Case' +Enter Your Name In ALL Upper Case: Thomas Lee +PS C:\> $Answer +Thomas Lee +PS C:\> $Answer = $Answer.ToUpper() # convert to all upper case. +PS C:\> $Answer +THOMAS LEE +``` + +## Strings are Immutable in .NET + +In .NET and PowerShell, a string is immutable +Once created, you can't change a System.String in memory after you define it. + +If you assign a string variable a new value (the old value plus a character), .NET creates an all-new string with same name) and marks the older string as out of scope and available for garbage collection. +This is generally not an issue in cases such as wanting to ensure user input is all upper-case. + +But if you have a script that makes a very large number of changes to any `System.String` object, you could encounter performance issues. +In such cases, you can use the .NET `System.Text.StringBuilder` class representaing mutable string of characters. +This class can provide significant performance gains in such scenarios. +For more information on the `StringBuilder` class, see [StringBuilder Class documentation page](https://docs.microsoft.com/dotnet/api/system.text.stringbuilder) +I plan to do another blog post on the differences. + +## Strings and Methods + +.NET strings also have other methods, including `ToLower()` that change a string to all lower case. +You can always discover the available methods of a string (or any other variable type) by piping the variable to `Get-Member`. +Like this: + +```powershell-console +PS C:\ $Answer | Get-Member -MemberType Method + + TypeName: System.String + +Name MemberType Definition +---- ---------- ---------- +Clone Method System.Object Clone(), System.Object ICloneable.Clone() +CompareTo Method int CompareTo(System.Object value), int CompareTo(string strB), int IComparabl… +Contains Method bool Contains(string value), bool Contains(string value, System.StringComparis… +CopyTo Method void CopyTo(int sourceIndex, char[] destination, int destinationIndex, int cou… +EndsWith Method bool EndsWith(string value), bool EndsWith(string value, System.StringComparis… +EnumerateRunes Method System.Text.StringRuneEnumerator EnumerateRunes() +Equals Method bool Equals(System.Object obj), bool Equals(string value), bool Equals(string … +GetEnumerator Method System.CharEnumerator GetEnumerator(), System.Collections.IEnumerator IEnumera… +GetHashCode Method int GetHashCode(), int GetHashCode(System.StringComparison comparisonType) +GetPinnableReference Method System.Char&, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, Public… +GetType Method type GetType() +GetTypeCode Method System.TypeCode GetTypeCode(), System.TypeCode IConvertible.GetTypeCode() +IndexOf Method int IndexOf(char value), int IndexOf(char value, int startIndex), int IndexOf(… +IndexOfAny Method int IndexOfAny(char[] anyOf), int IndexOfAny(char[] anyOf, int startIndex), in… +Insert Method string Insert(int startIndex, string value) +IsNormalized Method bool IsNormalized(), bool IsNormalized(System.Text.NormalizationForm normaliza… +LastIndexOf Method int LastIndexOf(char value), int LastIndexOf(char value, int startIndex), int … +LastIndexOfAny Method int LastIndexOfAny(char[] anyOf), int LastIndexOfAny(char[] anyOf, int startIn… +Normalize Method string Normalize(), string Normalize(System.Text.NormalizationForm normalizati… +PadLeft Method string PadLeft(int totalWidth), string PadLeft(int totalWidth, char paddingCha… +PadRight Method string PadRight(int totalWidth), string PadRight(int totalWidth, char paddingC… +Remove Method string Remove(int startIndex, int count), string Remove(int startIndex) +Replace Method string Replace(string oldValue, string newValue, bool ignoreCase, cultureinfo … +Split Method string[] Split(char separator, System.StringSplitOptions options), string[] Sp… +StartsWith Method bool StartsWith(string value), bool StartsWith(string value, System.StringComp… +Substring Method string Substring(int startIndex), string Substring(int startIndex, int length) +ToBoolean Method bool IConvertible.ToBoolean(System.IFormatProvider provider) +ToByte Method byte IConvertible.ToByte(System.IFormatProvider provider) +ToChar Method char IConvertible.ToChar(System.IFormatProvider provider) +ToCharArray Method char[] ToCharArray(), char[] ToCharArray(int startIndex, int length) +ToDateTime Method datetime IConvertible.ToDateTime(System.IFormatProvider provider) +ToDecimal Method decimal IConvertible.ToDecimal(System.IFormatProvider provider) +ToDouble Method double IConvertible.ToDouble(System.IFormatProvider provider) +ToInt16 Method short IConvertible.ToInt16(System.IFormatProvider provider) +ToInt32 Method int IConvertible.ToInt32(System.IFormatProvider provider) +ToInt64 Method long IConvertible.ToInt64(System.IFormatProvider provider) +ToLower Method string ToLower(), string ToLower(cultureinfo culture) +ToLowerInvariant Method string ToLowerInvariant() +ToSByte Method sbyte IConvertible.ToSByte(System.IFormatProvider provider) +ToSingle Method float IConvertible.ToSingle(System.IFormatProvider provider) +ToString Method string ToString(), string ToString(System.IFormatProvider provider), string IC… +ToType Method System.Object IConvertible.ToType(type conversionType, System.IFormatProvider … +ToUInt16 Method ushort IConvertible.ToUInt16(System.IFormatProvider provider) +ToUInt32 Method uint IConvertible.ToUInt32(System.IFormatProvider provider) +ToUInt64 Method ulong IConvertible.ToUInt64(System.IFormatProvider provider) +ToUpper Method string ToUpper(), string ToUpper(cultureinfo culture) +ToUpperInvariant Method string ToUpperInvariant() +Trim Method string Trim(), string Trim(char trimChar), string Trim(Params char[] trimChars) +TrimEnd Method string TrimEnd(), string TrimEnd(char trimChar), string TrimEnd(Params char[] … +TrimStart Method string TrimStart(), string TrimStart(char trimChar), string TrimStart(Params c… +``` + +If you look carefully at this list, you can see methods that convert a string to different kinds of numbers. +These methods would help you convert the string of 2 characters (e.g. 42) into an integer number. +That could well be the subject of another article that shows you how to achieve this. + +## Summary + +Turning the Caps Lock key on is not something I know how to do. +And if you did, it might confuse the user, for example if she sees the Caps Lock indicator light up on their keyboard. +As well, you would need to turn it off afterwards. + +Rather then depending on any user to always do the right thing, you can always ensure that the input is indeed in all upper case. +Never trust user input without validating it first. + +## Tip of the Hat + +This article is based on an earlier article here: [Can I Enable the Caps Lock Key?](https://devblogs.microsoft.com/scripting/can-i-enable-the-caps-lock-key/). +I re-developed the article around PowerShell. diff --git a/Posts/2021/03/File-System-Watcher-Engine-Event.md b/Posts/2021/03/File-System-Watcher-Engine-Event.md new file mode 100644 index 00000000..87a6fcc1 --- /dev/null +++ b/Posts/2021/03/File-System-Watcher-Engine-Event.md @@ -0,0 +1,116 @@ +--- +post_title: A Reusable File System Event Watcher for PowerShell +username: msn +categories: PowerShell +tags: PowerShell, FileSystemWatcher, Module +featured_image: +summary: FSWatcherEngineEvent module provides events of file system changes as powershell engine event +--- + +# A Reusable File System Event Watcher for PowerShell + +Some time ago I wanted to sync files from a source directory to a destination directory immediately after they had changed in the source directory. +As a C# developer I'm aware of a .Net Framework class named '[FileSystemWatcher](/dotnetapi/system.io.filesystemwatcher)' which suits this job perfectly. +A file system watcher listens to change notifications generated by the operating system and invokes a given function if the file change matches several filter criteria like the directory, the file name or the type of the change. +There are already many examples on the internet showing how to create and configure the watcher in PowerShell but this isn't something I can easily recall from memory at the moment I need it. +I made the FSWatcherEngineEvent PowerShell module to make these file system watchers easier to use. +It hides the C#-API behind a PowerShell command with argument completion, it keeps track of the created watchers, and provides commands to pause notifications and to clean up the watchers if they are no longer needed. + +After you install and import the module, you can create a new filesystem watcher. +As an example, you can watch for changes in directory 'C:\Temp\files'. +The command allows to specify the same parameters (with the same names) as if you are using the C# class directly. +This includes: + +- NotifyFilter: what kind of change triggers an event (by default: LastWrite, FileName, DirectoryName) +- Filter: a wildcard to define a subset of files to watch +- IncludeSubdirectory: extends the area of observation to the subdirectories of the specified path + +Please refer to the Microsofts reference documentation of the FileSystemWatcher class for the details. + +```powershell +New-FileSystemWatcher -SourceIdentifier "MyEvent" -Path C:\Temp\files +``` + +The watcher now sends notifications to PowerShells engine event queue using the source identifier "MyEvent". +You can consume the event by registering an event handler for the same source identifier. +The following example just writes the whole event converted to JSON to the console: + +```powershell-console +PS> Register-EngineEvent -SourceIdentifier "MyEvent" -Action { $event | ConvertTo-Json | Write-Host } + +Id Name PSJobTypeName State HasMoreData Location Command +-- ---- ------------- ----- ----------- -------- ------- +1 MyEvent NotStarted False $event|ConvertTo-Json|Wr… +``` + +PowerShell allows you to register more than one handler for a single source identifier but the FSWatcherEngineEvent module doesn't allow you to create more than one watcher using the same source identifier. + +To produce a new event, just write some characters to a file in the watched directory: + +```powershell-console +PS> "XYZ" >> C:\Temp\files\xyz + +{ + "ComputerName": null, + "RunspaceId": "b92c271b-c147-4bd6-97e4-ffc2308a1fcc", + "EventIdentifier": 4, + "Sender": { + "NotifyFilter": 19, + "Filters": [], + "EnableRaisingEvents": true, + "Filter": "*", + "IncludeSubdirectories": false, + "InternalBufferSize": 8192, + "Path": "D:\\tmp\\files\\", + "Site": null, + "SynchronizingObject": null, + "Container": null + }, + "SourceEventArgs": null, + "SourceArgs": null, + "SourceIdentifier": "MyEvent", + "TimeGenerated": "2021-03-13T21:39:50.4483088+01:00", + "MessageData": { + "ChangeType": 4, + "FullPath": "C:\\temp\\files\\xyz", + "Name": "xyz" + } +} +``` + +Events raised before a handler is registered remain in the queue until you consume them using '[Get-Event](xref:Microsoft.PowerShell.Utility.Get-Event)'/'[Remove-Event](xref:Microsoft.PowerShell.Utility.Remove-Event)'. + +To suspend the notification temporarily and to resume it later the following two commands can be used: + +```powershell +Suspend-FileSystemWatcher -SourceIdentifier "MyEvent" + +Resume-FileSystemWatcher -SourceIdentifier "MyEvent" +``` + +To keep track of all the filesystem watchers created in the current Powershell process, you can use the command 'Get-FileSystemWatcher': + +```powershell-console +PS> Get-FileSystemWatcher + +SourceIdentifier : MyEvent +Path : C:\Temp\files\ +NotifyFilter : FileName, DirectoryName, LastWrite +EnableRaisingEvents : True +IncludeSubdirectories : False +Filter : * +``` + +This command writes a state object to the pipe containing the configuration of all filesystem watchers. +Finally, if you want get rid of filesystem watchers the command 'Remove-FileSystemWatcher' disposes a filesystem watcher specified by the source identifier or by piping in a collection of watchers: + +```powershell +# to dispose one watcher +Remove-FileSystemWatcher -SourceIdentifier "MyEvent" + +# to dispose all +Get-FileSystemWatcher | Remove-FileSystemWatcher +``` + +Piping the filesystem watchers also works with the other commands. +If you like the module and want to see more you may inspect or fork its code at [github](https://github.com/wgross/fswatcher-engine-event). diff --git a/Posts/2021/03/ScriptingGuy.0006.readingbottomup.md b/Posts/2021/03/ScriptingGuy.0006.readingbottomup.md new file mode 100644 index 00000000..4d5ace0e --- /dev/null +++ b/Posts/2021/03/ScriptingGuy.0006.readingbottomup.md @@ -0,0 +1,156 @@ +--- +post_title: Reading a text file bottom up +username: tfl@psp.co.uk +Catagories: PowerShell +tags: Array +Summary: How can I read a file from the bottom up? +--- + +**Q:** I have a log file in which new data is appended to the end of the file. +That means the most recent entries are at the end of the file. +I’d like to be able to read the file starting with the last line and then ending with the first line, but I can’t figure out how to do that. + +**A:** There are loads of ways you can do this. +A simple way is to use the power of array handling in PowerShell. + +## The Get-Content Cmdlet + +Before getting into the solution, let's look at the `Get-Content` cmdlet. +The `Get-Content` cmdlet always reads a file from start to finish. +You can always get the very last line of the file like this: + +```powershell +Get-Content -Path C:\Foo\BigFile.txt | + Select-Object -Last 1 +``` + +This is similar to the `tail` command in Linux. +As is so often the case, the command doesn't quite do what you want it to. +That being said, with PowerShell 7, there's _always_ a way. + +## Using Arrays + +We can start by reading through the file from top to bottom. +Before displaying those lines to the screen we store them in an array, with each line in the file representing one element in the array. + +As you probably know, an array is a collection of objects. +An array can hold multiple objects of the same, or different, types. +In this case you want the array to hold the lines in your file. +Each line is a string. +Once you have the lines in the array, you can work backwards to achieve your goal. + +## Creating a simple file + +To demonstrate, let's start by creating a simple file, and output it to a local text file, like this + +```powershell-console +PS C:\Foo> $File = @' +>> violet +>> indigo +>> blue +>> green +>> yellow +>> orange +>> red +>> '@ +PS C:\Foo> $File | Out-File -Path C:\Foo\SmallFile.txt +PS C:\Foo> Get-ChildItem -Path C:\Foo\SmallFile.txt + + Directory: C:\Foo + +Mode LastWriteTime Length Name +---- ------------- ------ ---- +-a--- 22/01/2021 20:13 44 SmallFile.txt +``` + +Once you have created the file, you can get the contents and display it, like this: + +```powershell-console +PS C:\Foo> $Array = Get-Content -Path C:\Foo\SmallFile.txt +PS C:\Foo> $Array +violet +indigo +blue +green +yellow +orange +red +``` + +Admittedly, all we seem to have done so far is get back to where we started - displaying the file from the start to the finish not the reverse. +So how do we get to where you want to go? + +## Arrays vs text files + +There’s an important difference between a text file and an array. +With a text file, using `Get-Content`, you read it from only from the start to the finish. +Windows, .NET, and PowerShell do not provide a way to read the file in reverse. +However, once you have the file contained in an array. it’s easy to read the array from the bottom to the top. + +Let's start by working out how many lines there are in the array. +And, more as a sanity check, display how many lines there are in the file, like this: + +```powershell-console +PS C:\Foo> $Array = Get-Content -Path C:\Foo\SmallFile.txt +PS C:\Foo> $Length = $Array.count +PS C:\Foo> "There are $Length lines in the file" +There are 7 lines in the file +``` + +So that tells us you have the number of lines in the array that you expected. + +## Getting Array Members + +So let's give you a solution. +In our sample array, `$Array` we have 7 lines. +We can address any individual array member directly using `[<index>]` syntax (after the array name). +So the first item in the array always has an index number of 0 or `$Array[0]`). +In our array, the line **violet** has an index number of 0 so you can get to it using `$Array[0]`. +Likewise, red has an index number of 6, or `$Array[6]`. +But that doesn't help us much - just yet! + +A particularly neat feature of array handling in PowerShell is that we can work backwards in an array using negative index values. +An index of [-1] is always the last element of an array, [-2] is the penultimate line, and so on. +So `$Array[-1]` is Red, `$Array[-2]` is Orange, and so on. + +So what we do is to look first at `$Array[-1]`, then `$Array[-2]`, and so on, all withing a simple foreach loop, like this: + +```powershell-console +PS C:\Foo> $Array = Get-Content -Path C:\Foo\SmallFile.txt +PS C:\Foo> $Length = $Array.count +PS C:\Foo> "There are $Length lines in the file" +There are 7 lines in the file +PS C:\Foo> $Line = 1 +PS C:\Foo> 1..$Length | ForEach-Object {$Array[-$Line]; $Line++} +red +orange +yellow +green +blue +indigo +violet +``` + +This code snippet first sets a variable, `$Line`, to 1. +Then you read the file and display how many lines are in the file. +You then use `ForEach-Object` to run a script block once for each line in the file. +Inside the script block you get the array element starting at the end and output it to the console. +Then you increment the line number and repeat. + +This may be a little confusing if you haven't work with arrays, but once you get the hang of it, you see how simple it really is. +Arrays are a fantastic capability within PowerShell. + +## For More Information + +For more information on arrays in PowerShell, see [About_Arrays](https://docs.microsoft.com/powershell/module/microsoft.powershell.core/about/about_arrays). +And for more information on `Get-Content` see the [Get-Content](https://docs.microsoft.com/powershell/module/microsoft.powershell.management/get-content) help page. + +## Summary + +So as you saw, `Get-Content` does not read backwards through a file. +If you bring the file contents into an array, you can easily read it backwards. + +## Tip of the Hat + +This article is based on an earlier Scripting Guys blog article at [Can I Read a Text file from the Bottom Up?](https://devblogs.microsoft.com/scripting/can-i-read-a-text-file-from-the-bottom-up/). +I am not sure who wrote the original article. diff --git a/Posts/2021/03/ScriptingGuy.0007.folderexists.md b/Posts/2021/03/ScriptingGuy.0007.folderexists.md new file mode 100644 index 00000000..e0c94716 --- /dev/null +++ b/Posts/2021/03/ScriptingGuy.0007.folderexists.md @@ -0,0 +1,40 @@ +--- +post_title: Determine if a folder exists +username: baumanisf +Catagories: PowerShell +tags: File, Test-Path +Summary: How can I determine if a folder exists? +--- + +**Q:** Is there any way to determine whether or not a specific folder exists on a computer? +**A:** There are loads of ways you can do this. + +## The Test-Path Cmdlet + +The easiest way to do this is to use the `Test-Path` cmdlet. +It looks for a given path and returns `True` if it exists, otherwise it returns `False`. +You could evaluate the result of the `Test-Path` like in the code snippet below + +```powershell +$Folder = 'C:\\Windows' +"Test to see if folder [$Folder] exists" +if (Test-Path -Path $Folder) { + "Path exists!" +} else { + "Path doesn't exist." +} +``` +This is similar to the `-d $filepath` operator for IF statements in Bash. `True` is returned if `$filepath` exists, otherwise `False` is returned. + +## For More Information + +And for more information on `Test-Path` see the [Test-Path](https://docs.microsoft.com/powershell/module/microsoft.powershell.management/test-path) help page. + +## Summary + +So as you saw, `Test-Path` tests the existence of a path and returns a boolean value. +This return value can be evaluated in a IF statement for example. +## Tip of the Hat + +This article is based on an earlier Scripting Guys blog article at [How can I determine if a folder exists on a computer?](https://devblogs.microsoft.com/scripting/how-can-i-determine-if-a-folder-exists-on-a-computer/). +I am not sure who wrote the original article. diff --git a/Posts/2021/04/NewTemporaryFolders.md b/Posts/2021/04/NewTemporaryFolders.md new file mode 100644 index 00000000..b9f446a1 --- /dev/null +++ b/Posts/2021/04/NewTemporaryFolders.md @@ -0,0 +1,164 @@ +--- +post_title: Borrowing a built-in PowerShell command to create a temporary folder +username: sean.kearney@microsoft.com +Catagories: PowerShell, Function +tags: Function,Fun trick,Existing Cmdlet,New Purpose +Summary: Leveraging a built-in cmdlet in a new and interesting way +--- + +**Q:** Hey I have a question for you. It seems silly and I know I could probably put something together with Get-Random. But can you think of another way to create a temporary folder with a random name in PowerShell? + +Ideally, I'd like it to be in a user's own "Temporary Folder" is possible. + +**A:** We sure can! If Doctor Scripto was sitting here right now, I'd see that little green haired person shout out "Never fear, Scripto is here!" + +## New-TemporaryFile Cmdlet + +Within PowerShell there is a built in Cmdlet called `New-TemporaryFile`. Running this cmdlet simply creates a random 0 byte file in the `$ENV:Temp folder` in whichever platform you are working in. + +However, we can *borrow* the filename created and use it to create a folder instead. It’s not really difficult, but maybe just not thought of very often. + +When we execute the following cmdlet we get output similar to this as it generates a new 0 Byte random file in the User's Temp folder stored in `$ENV:Temp` + +```output +PS> New-TemporaryFile + +Mode LastWriteTime Length Name +---- ------------- ------ ---- +-a---- 3/31/2021 9:25 PM 0 tmpA927.tmp +``` + +Ok, that really wasn’t that impressive but what if we were to do this instead? + +```powershell +$File = New-TemporaryFile +``` + +Now we’ve created the file and stored it away in the `$File` object. With this we can remove the file of course using the `Remove-Item` cmdlet + +```powershell +Remove-Item -path $File -force +``` + +HA! I’ve already saved some time! The `$File` object is still there with the information I want to use. + +So, I could access the name in the object property and use it to create a directory instead in the following manner. + +```powershell +New-Item -ItemType Directory -Path $File.Name +``` + +But the problem is that it would be in whatever default folder PowerShell was looking into at the time. + +Hmmmmm…. How to solve that? + +But there is a built in variable called `$ENV:Temp` which targets the exact Temporary folder that the `New-TemporaryFile` cmdlet uses as well! + +I can then take that variable and the original name of the Temporary file and combine them together like this. + +```powershell +$ENV:Temp + '\' + $File.Name +``` + +*or* + +I can even put them together in a single String like this. + +```powershell +"$($ENV:Temp)\$($File.Name)" +``` + +With this I could just create a new temporary directory under our temp folder in this manner. + +```powershell +New-Item -ItemType Directory -Path "$($ENV:Temp)\$($File.Name)" +``` + +Now to identify where the file ended up, I could same thing as last time by storing it as an object like `$DirectoryName` if I wanted. Then I could remove the "Random Directory name" later if I needed to. + +```powershell +$Folder=New-Item -ItemType Directory -Path "$($ENV:Temp)\$($File.Name)" +``` + +Then when I am done with that folder that was presumably used to hold some garbage data. I can just use `Remove-Item` again. + +But because it's a directory, I need to add `-recurse -force` to ensure all data and Subfolders are removed. + +```powershell +Remove-Item -Path $Folder -Recurse -Force +``` + +But here is the fun and neat bit. If you needed on a regular basis, we could make this into a quick function for your code, module or to impress friends with! + +```powershell +Function New-TemporaryFolder { + # Create Temporary File and store object in $T + $File = New-TemporaryFile + + # Remove the temporary file .... Muah ha ha ha haaaaa! + Remove-Item $File -Force + + # Make a new folder based upon the old name + New-Item -Itemtype Directory -Path "$($ENV:Temp)\$($File.Name)" +} +``` + +Now at this point I had thought my journey was complete. It was until I posted the solution to the [Facebook group for the PowerShell Community Blog](https://www.facebook.com/groups/pscommunityblog/) to share. + +A fellow member of the Community noted the approach, while neat, was not very efficient. + +At that point I dug into the code on Github for the open source version of PowerShell 7.x to see how it was done there. + +In reading the source code for `New-TemporaryItem` I was able to see the .NET object being used to generate the file. It turns out there is also a .NET method that can be used to create just that temporary name which all I wanted to use in the first place for the directory name. + +When I ran this in the PowerShell Console it produced the following output of a New Temporary Folder + +```output +PS> [System.IO.Path]::GetTempFileName() +C:\Users\Administrator\AppData\Local\Temp\2\tmp3864.tmp</code></pre> +``` + +This was exactly what I wanted, that random temporary Name to be consumed for the `New-Item` Cmdlet. With this approach the function became a lot simpler and far more efficient! + +```powershell +Function New-TemporaryFolder { + # Make a new folder based upon a TempFileName + New-Item -ItemType Directory -Path([System.IO.Path]::GetTempFileName()) +} +``` + +But alas my victory was short lived. This method still created the file, it didn't just display the name. So the function ended up failing. + +But since really I just wanted that format to be used for the temporary directory. Plus the format for the temporary filename was as simple as `tmpxxxx.tmp` where the xxxx was a random hexadecimal number, I came up with a better idea! + +Just create a number between 0 and 65535 with `Get-Random` and use the `[convert]` accelerator to change it the 4 character Hexadecimal number instead. + +The end result looked like this and gave me the desired result I wanted. + +```output +PS> "$($Env:temp)\tmp$([convert]::tostring((get-random 65535),16).padleft(4,'0')).tmp" +C:\Users\doctorscripto\AppData\Local\Temp\tmp5633.tmp +``` + +Now I ended up with a working function that could produce the desired output I wanted and in a more efficient manner. + +```powershell +>Function New-TemporaryFolder { + # Make a new folder based upon a TempFileName + $T="$($Env:temp)\tmp$([convert]::tostring((get-random 65535),16).padleft(4,'0')).tmp" + New-Item -ItemType Directory -Path $T +} +``` + +Why did all of this pop into my head? I was actually creating some PowerShell for customer and needed a consistent and random set of folders in a common and easily erasable location. + +I was hoping that we had a `New-TemporaryDirectory` cmdlet, but found it was just as easy to write one by *borrowing* an existing cmdlet. + +It was fun as well to discover how I could improve on the solution by reading the [Source code for New-TemporaryItem](https://github.com/PowerShell/PowerShell/blob/master/src/Microsoft.PowerShell.Commands.Utility/commands/utility/NewTemporaryFileCommand.cs) on Github. + +Thanks to a little nudging from the Community. So a big Thank you to Joel Bennett for the critique! :) + +Sean Kearney - Customer Engineer/Microsoft - @PowerShellMan + +_*"Remember with great PowerShell comes great responsibilty..."*_ + diff --git a/Posts/2021/04/get-live-servers.md b/Posts/2021/04/get-live-servers.md new file mode 100644 index 00000000..4ee55376 --- /dev/null +++ b/Posts/2021/04/get-live-servers.md @@ -0,0 +1,161 @@ +--- +post_title: Testing the connection to computers in the Active Directory +username: tfl@psp.co.uk +Catagories: PowerShell +tags: Active Directory, networking +Summary: How can I get AD computers check to see they are online? +--- + +**Q:** As an administrator, I often have to do a lot of reporting on the servers in my domain. +Is there a simple way to test the connection to every server in my domain or every server or client host in a specific OU? + +**A:** Of course you can do this with PowerShell! You can use the Active Directory cmdlets and `Test-Connection`, although it is not as simple as one might like. + +## Using the `ActiveDirectory` module + +Microsoft has developed several modules to help you deploy and manage AD in your organisation or via Azure. +The `ActiveDirectory` module is one which Microsoft ships with Windows Server (although not installed by default). +You can also load the Remote Server Administration (RSAT) module for AD on a Windows 10 host. +The RSAT module allows you to manage the AD using PowerShell from a remote machine. +For more details on the `ActiveDirectory` module, see the [ActiveDirectory](https://docs.microsoft.com/powershell/module/addsadministration/) module documentation. + +Use the `Get-ADComputer` account to return details about some or all computers within the AD. +There are several ways to use `Get-ADComputer` to get just the computer accounts you want with any property you need. +These include using the **Identity** and **Filter** parameters. +Every computer account returned by `Get-ADComputer` contains two important properties: **Name** and **DNSHostName**. +The **Name** property is the single-label name of the computer (aka the NetBIOS name). +The **DNSHostName** property is the fully qualified DNS name for the computer. +Like this: + +```powershell-console +PS> Get-ADComputer -Filter * | Format-Table -Property Name, DNSHostName + +Name DNSHostName +---- ----------- +COOKHAM1 Cookham1.cookham.net +win10lt Win10LT.cookham.net +cookham24 cookham24.cookham.net +SLTPC sltpc.cookham.net +COOKHAM4LTDC Cookham4LTDC.cookham.net +``` + +So you might be tempted to think it simple to test connections to each computer. +You pipe the output of `Get-ADComputer` to `Test-Connection`, and it just works. +Sadly, it's not quite so simple. + +If you try this, here is what you would see: + +```powershell-console +PS> Get-ADComputer -Filter * | Test-Connection +Test-Connection: Cannot validate argument on parameter 'TargetName'. The argument is null, empty, or an element of the argument collection contains a null value. Supply a collection that does not contain any null values and then try the command again. +Test-Connection: Cannot validate argument on parameter 'TargetName'. The argument is null, empty, or an element of the argument collection contains a null value. Supply a collection that does not contain any null values and then try the command again. +Test-Connection: Cannot validate argument on parameter 'TargetName'. The argument is null, empty, or an element of the argument collection contains a null value. Supply a collection that does not contain any null values and then try the command again. +Test-Connection: Cannot validate argument on parameter 'TargetName'. The argument is null, empty, or an element of the argument collection contains a null value. Supply a collection that does not contain any null values and then try the command again. +Test-Connection: Cannot validate argument on parameter 'TargetName'. The argument is null, empty, or an element of the argument collection contains a null value. Supply a collection that does not contain any null values and then try the command again. +``` + +What is going on here? + +## Property/Parameter misalignment + +What we have here is a classic, albeit relatively uncommon, situation. +The `Test-Connection` cmdlet uses the parameter name **Target** to indicate the computer to which you are testing a connection. +However, in this pipelined command, the objects produced by `Get-ADComputer` do not contain properties of that name. +Instead, these objects have properties named **Name** and **DNSHostName**. + +[alert type="note" heading="Note"]With Windows PowerShell, you used the parameter **ComputerName** to indicate the computer you are investigating. +With PowerShell 7, the developers have changed this parameter name to **TargetName**. +For best compatibility, the cmdlet defines the**`ComputerName** alias to this parameter. +This cmdlet lets you use either **TargetName** or **Computername** with `Test-Connection`.[/alert] + +## ForEach-Object to the rescue + +It is pretty easy to get around this parameter/property alignment challenge. +You use the `Foreach-Object` cmdlet, like this: + +```powershell-console +PS> Get-ADComputer -Filter * | + ForEach-Object {"$_";Test-Connection -TargetName $_.Name;""} + +CN=COOKHAM1,OU=Domain Controllers,DC=cookham,DC=net + Destination: COOKHAM1 +Ping Source Address Latency BufferSize Status + (ms) (B) +---- ------ ------- ------- ---------- ------ + 1 cookham24 10.10.10.9 0 32 Success + 2 cookham24 10.10.10.9 0 32 Success + 3 cookham24 10.10.10.9 0 32 Success + 4 cookham24 10.10.10.9 0 32 Success + +CN=win10lt,OU=CookhamHQ,DC=cookham,DC=net + Destination: win10lt +Ping Source Address Latency BufferSize Status + (ms) (B) +---- ------ ------- ------- ---------- ------ + 1 cookham24 * 0 32 DestinationHost… + 2 cookham24 * 0 32 DestinationHost… + 3 cookham24 * 0 32 DestinationHost… + 4 cookham24 * 0 32 DestinationHost… + +CN=SLTPC,CN=Computers,DC=cookham,DC=net + Destination: SLTPC +Ping Source Address Latency BufferSize Status + (ms) (B) +|---- ------ ------- ------- ---------- ------ + 1 cookham24 2a02:8010:6386:0:f810:2b… 1 32 Success + 2 cookham24 2a02:8010:6386:0:f810:2b… 0 32 Success + 3 cookham24 2a02:8010:6386:0:f810:2b… 0 32 Success + 4 cookham24 2a02:8010:6386:0:f810:2b… 3 32 Success +etc +``` + +## Using the Extensible Type System + +If you plan to do a lot of this sort of work, there is a more straightforward way to get around this property/parameter alignment issue. +You can use the Extensible Type System (ETS) to extend any AD Computer object to contain an alias to the `Name` or `DNSHostName` property. +You define this extension via a small XML file which you then import, like this: + +```powershell-console +PS> Get-Content '.\aaatypes.types.ps1xml' +<Types> + <Type> + <Name>Microsoft.ActiveDirectory.Management.ADComputer</Name> + <Members> + <AliasProperty> + <Name>TargetName</Name> + <ReferencedMemberName>DNSHostName</ReferencedMemberName> + </AliasProperty> + </Members> + </Type> + +</Types> + +PS> Update-TypeData -PrependPath .\aaatypes.types.ps1xml +PS> Get-ADComputer -Identity Cookham1 | Test-Connection + + Destination: Cookham1.cookham.net + +Ping Source Address Latency BufferSize Status + (ms) (B) +---- ------ ------- ------- ---------- ------ + 1 cookham24 10.10.10.9 0 32 Success + 2 cookham24 10.10.10.9 0 32 Success + 3 cookham24 10.10.10.9 0 32 Success + 4 cookham24 10.10.10.9 0 32 Success +``` + +You can persist this ETS extension by adding the `Update-TypeData` to your PowerShell profile. +That way, every time you start a PowerShell session, that ETS extension is in place and ready to assist you. + +For details of and background to the ETS, see the [Extended Type System Overview](https://docs.microsoft.com/powershell/scripting/developer/ets/overview). + +## Summary + +The `Get-ADComputer` cmdlet produces objects whose properties the object developers have not aligned, pipeline wise, with `Test-Connection`. +There is a simple way around that, using `For-EachObject`, although it takes a bit more typing. +You can also use the ETS to extend the **ADComputer** object to have a more friendly alias. + +## Tip of the Hat + +This article was based on a request in this blog's issue queue +See the post [Request - How to get all the alive servers in the domain?](https://github.com/PowerShell/Community-Blog/issues/21) diff --git a/Posts/2021/04/tfl-IsUserALocalAdministrator.md b/Posts/2021/04/tfl-IsUserALocalAdministrator.md new file mode 100644 index 00000000..da3d7285 --- /dev/null +++ b/Posts/2021/04/tfl-IsUserALocalAdministrator.md @@ -0,0 +1,131 @@ +--- +post_title: Is a User A Local Administrator +username: tfl@psp.co.uk +Catagories: PowerShell +tags: local users, security, logon scripts +Summary: In a logon script, how you can tell if the user is a local administrator +--- + +**Q:** Some of the things we do in our logon scripts require the user to be a local administrator. How can the script tell if the user is a local administrator or not, using PowerShell 7. + +**A:** Easy using PowerShell 7 and the LocalAccounts module + +## Local Users and Groups + +The simple answer is of course, easily. +And since you ask, with PowerShell 7! +But let's begin lets begin by reviewing local users and groups in Windows. + +Every Windows system, except for Domain Controllers, maintains a set of local accounts - local users and local groups. +Domain controllers use the AD and do not really have local accounts as such. +You use these local accounts in addition to domain users and domain groups on domain-joined hosts when setting permissions. +You can logon to a given server using a local account or a domain account. +On Domain Controllers you can only login using a domain account. + +As with AD groups, local groups and local users each have a unique Security ID (SID). +When you give a local user or group access to a file or folder, Windows adds that SID to the object's Access Control List. +This is the same way Windows enables you to give permissions to a local file or folder to any Active DIrectory user or group. + +Additionally, Windows and some Windows features create "well known" local groups. +The intention is that you add users to these groups to enable those users to perform specific administrative functions on just those servers. + +Traditionally, you might have used the `Wscript.Network` COM object, in conjunction with ADSI. +You can, of course, use the older approach in side PowerShell 7, but why bother? +The good news with PowerShell 7, you can use the `Microsoft.PowerShell.LocalAccounts` module to manage local accounts. +At the time of writing, this is a Windows only module. + +## The Microsoft.PowerShell.LocalAccounts module + +In PowerShell 7 for Windows, you can use the `Microsoft.PowerShell.LocalAccounts` module to manage local users and group. +This module is a Windows PowerShell module which PowerShell 7 loads from `C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules\Microsoft.PowerShell.LocalAccounts`. + +This module contains 15 cmdlets, which you can view like this: + +```powershell-console +PS> Get-Command -Module Microsoft.PowerShell.LocalAccounts + +CommandType Name Version Source +----------- ---- ------- ------ +Cmdlet Add-LocalGroupMember 1.0.0.0 Microsoft.PowerShell.LocalAccounts +Cmdlet Disable-LocalUser 1.0.0.0 Microsoft.PowerShell.LocalAccounts +Cmdlet Enable-LocalUser 1.0.0.0 Microsoft.PowerShell.LocalAccounts +Cmdlet Get-LocalGroup 1.0.0.0 Microsoft.PowerShell.LocalAccounts +Cmdlet Get-LocalGroupMember 1.0.0.0 Microsoft.PowerShell.LocalAccounts +Cmdlet Get-LocalUser 1.0.0.0 Microsoft.PowerShell.LocalAccounts +Cmdlet New-LocalGroup 1.0.0.0 Microsoft.PowerShell.LocalAccounts +Cmdlet New-LocalUser 1.0.0.0 Microsoft.PowerShell.LocalAccounts +Cmdlet Remove-LocalGroup 1.0.0.0 Microsoft.PowerShell.LocalAccounts +Cmdlet Remove-LocalGroupMember 1.0.0.0 Microsoft.PowerShell.LocalAccounts +Cmdlet Remove-LocalUser 1.0.0.0 Microsoft.PowerShell.LocalAccounts +Cmdlet Rename-LocalGroup 1.0.0.0 Microsoft.PowerShell.LocalAccounts +Cmdlet Rename-LocalUser 1.0.0.0 Microsoft.PowerShell.LocalAccounts +Cmdlet Set-LocalGroup 1.0.0.0 Microsoft.PowerShell.LocalAccounts +Cmdlet Set-LocalUser 1.0.0.0 Microsoft.PowerShell.LocalAccounts +``` + +As you can tell, these cmdlets allow you to add, remove, change, enable and disable a local user or local group +And they allow you to add, remove and get the local group's members. +These cmdlets are broadly similar to the ActiveDirectory cmdlets, but work on local users. +And as noted above, you can use domain users/groups as a member of a local group should you wish or need to. + +You use the `Get-LocalGroupMember` command to view the members of a local group, like this: + +```powershell-console +PS> Get-LocalGroupMember -Group 'Administrators' + +ObjectClass Name PrincipalSource +----------- ---- --------------- +Group COOKHAM\Domain Admins ActiveDirectory +User COOKHAM24\Administrator Local +User COOKHAM\JerryG ActiveDirectory +User COOKHAM24\Dave Local +``` + +As you can see in this output, the local Administrators group on this host contains domain users and groups as well as local users + +## Is the User an Administrator? + +It's easy to get membership of any local group, as you saw above. +But what if you want to find out if a given user is a member of some local administrative group? +That too is pretty easy and take a couple of steps. +One way you can get the name of the current user is by using `whoami.exe`. +Then you can get the members of the local administrator's group. +Finally, you check to see if the currently logged on user is a member of the group. +All of which looks like this: + +```powershell-console +PS> # Get who I am +PS> $Me = whoami.exe +PS> $Me +Cookham\JerryG + +PS> # Get members of administrators group +PS> $Admins = Get-LocalGroupMember -Name Administrators | + Select-Object -ExpandProperty name + +PS> # Check to see if this user is an administrator and act accordingly +PS> if ($Admins -Contains $Me) { + "$Me is a local administrator"} + else { + "$Me is NOT a local administrator"} +Cookham\JerryG is a local administrator +``` + +If the administrative group contains user running the script, then `$Me` is a user in that local admin group. + +In this snippet, we just echo the fact that the user is, ir is not, a member of the local administrators group. +You can adapt it to ensure a user is a member of the appropriate group before attempting to run certain commands. +And you can also adapt it to check for membership in other local groups such as **Backup Operators** or **Hyper-V Users** which may be relevant. + +In your logon script, once you know that the user is a member of a local administrative group, you can carry out any tasks that require that membership. +And if the user is not a member of the group, you could echo that fact, and avoid using the relevant cmdlets. + +## Summary + +Using the Local Accounts module in PowerShell 7, it's easy to manage local groups! +You can, of course, manage the groups the same way in Windows PowerShell. + +## Tip of the Hat + +This article was originally a VBS based solution as described in (an earlier blog post)[https://devblogs.microsoft.com/scripting/how-can-i-determine-if-a-user-is-a-local-administrator/]. +I am not sure who the author of the original post was - but thanks. diff --git a/Posts/2021/05/SendingDataToTheClipBoard.md b/Posts/2021/05/SendingDataToTheClipBoard.md new file mode 100644 index 00000000..23970dff --- /dev/null +++ b/Posts/2021/05/SendingDataToTheClipBoard.md @@ -0,0 +1,203 @@ +--- +post_title: Sending data to the Clipboard from PowerShell +username: sean.kearney@microsoft.com +Categories: PowerShell, vbScript, Conversion, Cmdlet +tags: PowerShell, Clipboard, vbScript +Summary: Sending data to the clipboard from all versions of PowerShell +--- + +**Q:** Hey I have a fun question! I remember reading a while back about using +VBScript to paste to the clipboard. Are we able to do that with PowerShell? + +**A:** Why yes, yes we can! It is far often a much quicker solution if we +start with PowerShell! + +## Pasting content to the clipboard, the old VBScript method + +Before we show the quick and easy solution, let's learn how we could adapt an +older solution. + +Now back in the day if I wanted to paste something on the clipboard I would go +down to the store, get some glue and.... + +"DOH! Wrong Clipboard!" (Knew I should have splashed some water on my face +before typing this up!) + +What I should have said was back before PowerShell existed we actually had TWO +methods to paste text data to the clipboard. + +One was a nice simple solution if you were working in DOS or has simple text +output from a VBScript. You would pipe the output to the `clip` command as +seen below + +```output +dir | clip +``` + +Once this was complete you could paste your captured text using a `CTRL-V` in +whichever Windows application. + +Another method that presented itself was using some code such as this in +VBScript + +```VBScript +strCopy = "This text has been copied to the clipboard." +Set objIE = CreateObject("InternetExplorer.Application") +objIE.Navigate("about:blank") +objIE.document.parentwindow.clipboardData.SetData "text", strCopy +objIE.Quit +``` + +So I could re-use this solution in PowerShell quite easily. I do this in case +you might ever see some older VBScript that you might want to reuse. + +The first line in VBScript is assigning a string + +```VBScript +strCopy = "This text has been copied to the clipboard." +``` + +In PowerShell I can do this in the following manner. + +```powershell +$strCopy = "This text has been copied to the clipboard." +``` + +The next line is where a Comobject is created. + +```VBScript +Set objIE = CreateObject("InternetExplorer.Application") +``` + +The equivalent code in PowerShell to do the same thing and even use the same +variable name for the object would be + +```powershell +# Create a connect to Internet Explorer and launch it as a +# hidden application +$objIE = New-Object -comobject "InternetExplorer.Application" +``` + +Then from this point the final lines are just manipulating data in the Object. + +```VBScript +objIE.Navigate("about:blank") +objIE.document.parentwindow.clipboardData.SetData "text", strCopy +objIE.Quit +``` + +Which in PowerShell would look like this. + +```powershell +# Point Internet Explorer to "Blank page" +$objIE.Navigate("about:blank") + +# Leverage Internet explorer to send content to the clipboard +$objIE.document.parentwindow.clipboardData.SetData("text", $strCopy) + +# A job well done! Quit and go back home +$objIE.Quit +``` + +However if you try this solution in a modern version of Windows, it will appear +to just sit and hang in PowerShell. + +We can one extra line to the original code and you see why. + +```powershell +$objIE.Navigate("about:blank") + +# Show the hidden Internet Explorer background application +$objIE.Visible=$True + +$objIE.document.parentwindow.clipboardData.SetData("text", strCopy) +$objIE.Quit +``` + +The following Window below demonstrates why the old solution, even when +converted to PowerShell failed. + +![Prompt To Allow or Deny Clipboard Paste in Internet Explorer](./media/SendingDataToTheClipBoard/InteractivePromptStoppingTheOldSolution.PNG) + +In fact, even if we just ran it in vbScript today, it would have failed in an +equal manner. + +## The drawback to converting from VBScript + +So that was pretty cool, you tried to re-use some VBScript to meet your task. +In this case because security has improved in past 17 years, Internet Explorer +is not allowed to just paste things to the clipboard. + +But although this is a nice way to learn how to convert over some older code +from VBScript, it is actually not a good used of PowerShell for two reasons. + +1. It leverages Internet Explorer which, for this purpose, is a big resource +to solve the problem at hand. We can also no longer automate in this manner. +1. PowerShell has built in cmdlets to solve the problem which are far easier +to use. They not only work well in Windows PowerShell, but also just as +seamlessly across all supported Operating Systems when using PowerShell 7.x + +## The clipboard cmdlets in PowerShell + +You can verify they exist by just using `Get-Command` like the sample below + +```output +Get-Command *clipboard* + +CommandType Name Version Source +----------- ---- ------- ------ +Cmdlet Get-Clipboard 7.0.0.0 Microsoft.PowerShell.Management +Cmdlet Set-Clipboard 7.0.0.0 Microsoft.PowerShell.Management +``` + +To populate the clipboard with a directory structure, as an example, I can +execute the following line + +```output +PS> Get-Childitem | Set-Clipboard +``` + +There is no visual output because the data is now stored on the Clipboard. + +To verify this in PowerShell you can use the `Get-Clipboard` Cmdlet. + +```output +PS> Get-Clipboard +C:\Demo\AzureADBaseline +C:\Demo\AzureDSC +C:\Demo\AzureVM-Json +C:\Demo\DualStateMitigate +C:\Demo\testmoduleforme.ps1 +C:\Demo\TheShellofBlueness.docx +``` + +So yes, I wasted a bit of time showing you how to convert some VBScript. +Please have pity on me for my intent was good. Shame on me. :) + +## Summary + +It is nice to know that you can convert over to PowerShell in a pretty +simple manner from vbScript if you needed to. There are a lot of excellent +examples of how to manage Windows environments with VBScript readily written. + +It is equally important to understand why we would choose to start fresh with +the solution in PowerShell. + +It gives us a solution, in this case of manipulating the clipboard in an +Operating System; which is consistent across the board whether you choose to +use Windows, MacOS, Linux or any supported Operating System for PowerShell 7.x. + +The choice is yours my friends! + +## Tip of the Hat + +This article was based on one written back on older post on +["Can I Copy Script Output to the Clipboard"](https://devblogs.microsoft.com/scripting/can-i-copy-script-output-to-the-clipboard/) + +I do not recall the author but it was a good way to learn how to +programmatically set the clipboard back then. It was time to get it updated. + +Cheers all! Your friend in Automation +Sean Kearney - Customer Engineer/Microsoft - @PowerShellMan + +_"Remember with great PowerShell comes great responsibilty..."_ diff --git a/Posts/2021/05/media/SendingDataToTheClipBoard/InteractivePromptStoppingTheOldSolution.PNG b/Posts/2021/05/media/SendingDataToTheClipBoard/InteractivePromptStoppingTheOldSolution.PNG new file mode 100644 index 00000000..b18bda58 Binary files /dev/null and b/Posts/2021/05/media/SendingDataToTheClipBoard/InteractivePromptStoppingTheOldSolution.PNG differ diff --git a/Posts/2021/05/tfl-WMIEventHandler.md b/Posts/2021/05/tfl-WMIEventHandler.md new file mode 100644 index 00000000..e96b017a --- /dev/null +++ b/Posts/2021/05/tfl-WMIEventHandler.md @@ -0,0 +1,371 @@ +--- +post_title: How Do I Discover Changes to an AD Group's Membership +username: tfl@psp.co.uk +Catagories: PowerShell +tags: AD, WMI, WMI Eventing +Summary: How to create a permanent event handler to detect changes in an AD Group +--- + +**Q:** Is there an easy way to detect and changes to important the membership of AD Groups? + +**A:** Easy using PowerShell 7, WMI, and the CIM Cmdlets. + +## WMI + +Windows Management Instrumentation (WMI) is an important component of the Windows operating system. +WMI is an infrastructure of both management data and management operations on Windows-based +computers. You can use PowerShell to retrieve information about your host, such as the BIOS Serial +number. Additionally, you can perform management actions, such as creating an SMB share. + +WMI is, in many cases, just another way to do things. For example, you can use WMI to create an SMB +share by using the `Create` method of the **Win32_Share** class. For more information, see the +documentation for the +[Win32_Share class](https://docs.microsoft.com/windows/win32/cimwin32prov/win32-share). In most +cases, you use PowerShell cmdlets, such as the SMB cmdlets, to manage your SMB shares. The value of +WMI is that it can provide you access to more information and features that are not available using +cmdlets. + +In writing this article, I assume you have an understanding of WMI. In specific, I assume you +understand WMI namespaces, classes, properties, and methods. If not, you might like to look at the +[WMI Documentation](https://docs.microsoft.com/windows/win32/wmisdk/wmi-start-page). + +## WMI Eventing + +A cool and very powerful feature of WMI is eventing. With WMI eventing, you can subscribe to an +event, such as the change of an AD group's membership. If and when that event occurs, you can take +some action, such as writing to a log file or sending an email. WMI event handling is fairly +straightforward and very powerful - if you know what classes to use and how to use them! + +There are two broad types of eventing within WMI. With temporary eventing, you use PowerShell inside +a PowerShell session to subscribe to the events and process them. If you close that session, the +event subscriptions and event handlers are lost. To enable temporary WMI event monitoring to +continue, you must leave the host turned on and logged in, which may be a suboptimal situation. +Temporary event handling can be great for troubleshooting but not ideal for longer-term monitoring. + +With permanent event handling, you also tell WMI what events to do and what to do when they occur. +To do that, you add the details of event handling to the WMI repository. By doing so, WMI can +continue to monitor the event after close your session, logoff, or even reboot your host. And with +PowerShell and PowerShell remoting, it is pretty easy to deploy WMI event detection on multiple +servers. + +I warn you that the documentation for eventing may not be great in all cases. Some documentation is +focused on developers and thus lacks good PowerShell examples. + +## Permanent Event Consumers + +Within WMI, a permanent event consumer is a built-in COM component that does something when any +given event occurs. In theory, I suppose you could develop a private WMI event consumer, but I have +never seen one developed. I am not suggesting that someone has not done it, of course. If you have +seen this - please comment as I'd love to see the code and understand the details. + +There are five key WMI permanent event consumers which Microsoft provides within Windows: + +- **Active Script Consumer**: You use this to run a specific VBS script. +- **Log File Consumer**: This handler writes strings of customizable text to a text file. +- **NT Event Log Consumer**: This consumer writes event details into the Windows Application event + log. +- **SMTP Event Consumer**: You use this consumer to send an SMTP email message when an event occurs +- **Command Line Consumer**: This consumer runs a program with parameters, for example, run + PowerShell 7 and specific a script to run. + +The Active Script consumer _only_ runs VBS scripts. Short of redeveloping the COM component, you can +not use this consumer with PowerShell scripts. The Log File Consumer is excellent for writing short +highly-customised messages to a text file but can take some time and effort to implement. For most +IT Pros, the Command Line consumer is the one to choose. With this consumer, you get WMI to run a +PowerShell script any time an event occurs, such as a change to an AD group. Let's look at how you +use this permanent event consumer to discover changes to the membership of the Enterprise Admins +group. + +## Creating a permanent event handler + +With WMI permanent event handling, you need to create three objects within the CIM database + +- Event filter - the filter tells WMI which event to detect, such as a change in the change to an AD + group. +- Event consumer - this tells WMI which permanent event consumer to run and how to invoke the + consumer, such as to run the Command Line consumer and run `Monitor.ps1`. +- Event binding - this binds the filter (what event to look out for) to the consumer (what to do + when the event occurs happens) + +To carry out these three operations, you inserting new objects into three specific WMI system +classes. The WMI system class instances enable WMI to continue to process events after you stop your +PowerShell session, log off, or restart your host. + +In the code below, you use the Command Line consumer to detect changes to the AD's Enterprise Admins +group. Every time the change event occurs, you want WMI to run a specific script, namely +`Monitor.ps1`. This script displays a list of the current members of the **Enterprise Admins** group +to a log file and reports whether the membership now contains unauthorized users. If the script +finds that an unauthorized user is now a group member, it writes details to a text file for you to +review later. + +## The Solution + +There are several steps in this solution. +So please, fasten your seat belts, and away we go. + +### Setting up + +In this post, you want to detect whether an unauthorized user is a member of the Enterprise Admins +group. You must first create a file of authorized users. Then you create two helper functions to +assist you in testing the code. The function to delete all aspects of the WMI event filter from your +host is useful unless you plan to keep the filter running forever. + +```powershell +# 1. Create a list of valid users for the Enterprise Admins group +$OKUsersFile = 'C:\\Foo\\OKUsers.Txt' +$OKUsers = @' +Administrator +JerryG +'@ +$OKUsers | + Out-File -FilePath $OKUsersFile + +# 2. Define two helper functions to get/remove permanent events +Function Get-WMIPE { + '*** Event Filters Defined ***' + Get-CimInstance -Namespace ROOT\\subscription -ClassName __EventFilter | + Where-Object Name -eq "EventFilter1" | + Format-Table Name, Query + '***Consumer Defined ***' + $NS = 'ROOT\\subscription' + $CN = 'CommandLineEventConsumer' + Get-CimInstance -Namespace $ns -ClassName $CN | + Where-Object {$_.name -eq "EventConsumer1"} | + Format-Table Name, CommandLineTemplate + '***Bindings Defined ***' + Get-CimInstance -Namespace ROOT\\subscription -ClassName __FilterToConsumerBinding | + Where-Object -FilterScript {$_.Filter.Name -eq "EventFilter1"} | + Format-Table Filter, Consumer +} +Function Remove-WMIPE { + Get-CimInstance -Namespace ROOT\\subscription __EventFilter | + Where-Object Name -eq "EventFilter1" | + Remove-CimInstance + Get-CimInstance -Namespace ROOT\\subscription CommandLineEventConsumer | + Where-Object Name -eq 'EventConsumer1' | + Remove-CimInstance + Get-CimInstance -Namespace ROOT\\subscription __FilterToConsumerBinding | + Where-Object -FilterScript {$_.Filter.Name -eq 'EventFilter1'} | + Remove-CimInstance +} +``` + +These two steps produce no output. +When you create the `OkUsers.txt` file - ensure the users in the file are actually in your AD. + +### Create a WQL event query and WMI event filter + +To tell WMI what event you want WMI to detect, you create a WMI Query Language (WQL) query. In each +WMI namespace, you can find various system classes representing event notification. You can use the +**__InstanceModificationEvent** class, for example, to detect any modification of an instance (in +that namespace). You can likewise use the **__MethodInvocationEvent** class to track WMI method +invocations. If things change anywhere in a Windows host, you can probably use a WMI event to detect +the change. + +Here's the code to create the WQL query and the WMI event filter + +```powershell +# 3. Create a WQL event filter query +$Group = 'Enterprise Admins' +$Query = @" + SELECT * From __InstanceModificationEvent Within 10 + WHERE TargetInstance ISA 'ds_group' AND + TargetInstance.ds_name = '$Group' +"@ + +# 4. Create the event filter +$Param = @{ + QueryLanguage = 'WQL' + Query = $Query + Name = 'EventFilter1' + EventNameSpace = 'ROOT/directory/LDAP' +} +$IHT = @{ + ClassName = '__EventFilter' + Namespace = 'ROOT/subscription' + Property = $Param +} +$InstanceFilter = New-CimInstance @IHT +``` + +In this code (which produces no output), the filter query does not state which namespace the query +is looking at, just that there is a target class for WMI to monitor. In the event filter, you create +a new occurrence in the **EventFilter** class in the **ROOT/Subscription** namespace. This +occurrence tells WMI to monitor the **ROOT/directory/LDAP** namespace for the **ds_group** class. + +### Creating the Event Consumer + +The next step is to create an event consumer - what you want WMI to do when it detects the event has +occurred. In our example, you want the WMI permanent event handler COM object to run a script +`Monitor.ps1` any time the event occurs. So whenever WMI detects a change to the Enterprise admins +group, you want WMI to run the script. + +```powershell +# 5. Create Monitor.ps1 +$MONITOR = @' +$LogFile = 'C:\\Foo\\Grouplog.Txt' +$Group = 'Enterprise Admins' +"On: [$(Get-Date)] Group [$Group] was changed" | + Out-File -Force $LogFile -Append -Encoding Ascii +$ADGM = Get-ADGroupMember -Identity $Group +# Display who's in the group +"Group Membership" +$ADGM | Format-Table Name, DistinguishedName | + Out-File -Force $LogFile -Append -Encoding Ascii +$OKUsers = Get-Content -Path C:\\Foo\\OKUsers.txt +# Look at who is not authorized +foreach ($User in $ADGM) { + if ($User.SamAccountName -notin $OKUsers) { + "Unauthorized user [$($User.SamAccountName)] added to $Group" | + Out-File -Force $LogFile -Append -Encoding Ascii + } +} +"**********************************`n`n" | +Out-File -Force $LogFile -Append -Encoding Ascii +'@ +$MONITOR | Out-File -Path C:\\Foo\\Monitor.ps1 + +# 6. Create a WMI event consumer +# The consumer runs PowerShell 7 to execute C:\\Foo\\Monitor.ps1 +$CLT = 'Pwsh.exe -File C:\\Foo\\Monitor.ps1' +$Param =[ordered] @{ + Name = 'EventConsumer1' + CommandLineTemplate = $CLT +} +$ECHT = @{ + Namespace = 'ROOT/subscription' + ClassName = "CommandLineEventConsumer" + Property = $param +} +$InstanceConsumer = New-CimInstance @ECHT +``` + +The monitoring script is fairly simple - each time the event occurs, it prints some information to a +log file. Then it looks to see if the Enterprise Admins group contains unauthorized users - and if +so, the script reports that fact to the log file. This script is fairly simple, and you can +embellish. as needed. You could, for example, remove all unauthorized users. + +To create a WMI event consumer, you add a new occurrence to the **CommandLineEventConsumer** class +within the namespace **ROOT/Subscription**. + +### Binding the Event Filter and the Event Consumer + +With the event filter and event consumer details added to WMI, you need to bind the two - telling +WMI to detect THAT event and when it occurs, run THIS script. You could pre-create, for example, +multiple event filters and event consumers. Once the binding is in place, WMI starts the monitoring +process. + +```powershell +# 7. Bind the filter and consumer +$Param = @{ + Filter = [ref]$InstanceFilter + Consumer = [ref]$InstanceConsumer +} +$IBHT = @{ + Namespace = 'ROOT/subscription' + ClassName = '__FilterToConsumerBinding' + Property = $Param +} +$InstanceBinding = New-CimInstance @IBHT +``` + +### Checking your work + +A great way to check your work is to call the `Get-WMIPE` function you created earlier. What you +should see is: + +```powershell-console +PS > # 8. Viewing the event registration details +PS > Get-WMIPE +*** Event Filters Defined *** + +Name Query +---- ----- +EventFilter1 SELECT * From __InstanceModificationEvent Within 10 + WHERE TargetInstance ISA 'ds_group' AND + TargetInstance.ds_name = 'Enterprise Admins' + +***Consumer Defined *** + +Name CommandLineTemplate +---- ------------------- +EventConsumer1 Pwsh.exe -File C:\\Foo\\Monitor.ps1 + +***Bindings Defined *** + +Filter Consumer +------ -------- +__EventFilter (Name = "EventFilter1") CommandLineEventConsumer (Name = "EventConsumer1") + +``` + +### Testing your work + +So having created the event query, the event filter, the event consumer and the filter to consumer +binding, you can test your work. The easiest way to test this is to add a new user to the group. +Then, wait a few seconds for WMI to process the event, then look at the output. If everything is +working correctly, you should see this output: + +```powershell-console +PS > # 9. Adding a user to the Enterprise Admins group +PS > Add-ADGroupMember -Identity 'Enterprise admins' -Members Malcolm +PS > +PS > # 10. Viewing the Grouplog.txt file +PS > Get-Content -Path C:\\Foo\\Grouplog.txt +On: [04/20/2021 15:41:49] Group [Enterprise Admins] was changed + +Name DistinguishedName +---- ----------------- +Malcolm CN=Malcolm,OU=IT,DC=Reskit,DC=Org +Jerry Garcia CN=Jerry Garcia,OU=IT,DC=Reskit,DC=Org +Administrator CN=Administrator,CN=Users,DC=Reskit,DC=Org + +Unauthorized user [Malcolm] added to Enterprise Admins +********************************** +``` + +### Troubleshooting + +This code, of course should "just work". If not, you need to perform troubleshooting and here are +three things to look for: + +- Is the WQL query correct? +- Are the event and subscriptiong classes in the namespace(s) you think it is in? +- Is the `Monitor.ps1` script doing what you actually wanted? + +The **Microsoft-Windows-WMI-Activity/Operational** event log can be useful in tracking down issues. +And if you get stuck, feel free to visit the +[Spiceworks PowerShell forum](https://community.spiceworks.com/programming/powershell). + +### Tidying up + +After you play with a WMI filter like this, make sure you clean up. You probably don't want the +filter to run forever, so remove it as soon as you can. To remove it, invoke the `Remove-WMIPE` +function. And you should probably remove any inappropriate users from the Enterprise Admins group + +```powershell +# 11. Tidying up +Remove-WMIPE # invoke this function you defined above +$RGMHT = @{ + Identity = 'Enterprise Admins' + Member = 'Malcolm' + Confirm = $false +} +Remove-ADGroupMember @RGMHT +``` + +This step creates no output. You might wish to call `Get-WMIPE` again to verify you have removed all +three class occurrences. + +## Summary + +WMI eventing is very powerful and straightforward to implement. There are thousands of WMI events +you could subscribe to and which may help troubleshooting activities. In this case, you are +examining unauthorized changers to an AD group. The WMI documentation does not provide a definitive +guide to the events you might be interested in - at least that I can find. + +For some more details on using WMI in PowerShell 7, see my recently published +[PowerShell 7 book](https://www.wiley.com/en-gb/PowerShell+7+for+IT+Professionals-p-9781119644705). +I devote chapter 9 to WMI and using the CIM cmdlets. You can find the scripts from this blog post +and that chapter in my +[GitHub repository](https://github.com/doctordns/Wiley20/tree/master/09%20-%20WMI). diff --git a/Posts/2021/05/tfl-output_as_string.md b/Posts/2021/05/tfl-output_as_string.md new file mode 100644 index 00000000..a7b3e98f --- /dev/null +++ b/Posts/2021/05/tfl-output_as_string.md @@ -0,0 +1,209 @@ +--- +post_title: How to send output to a file +username: tfl@psp.co.uk +Categories: PowerShell +tags: PowerShell, output +Summary: Multiple ways of sending output to a file. +--- + +**Q:** Is there an easy way to save my script output to a text file rather than displaying it on screen? + +**A:** Of course - there are multiple ways to do just that! + +## PowerShell and Output + +One of PowerShell's great features is the way it automatically formats output. +You type a command - PowerShell gives you the output it thinks you want. +If the default output is not what you need, use the formatting cmdlets like `Format-Table` and `Format-List` to get what you want. +But sometimes, what you want is getting output to a file, not to the console. +You might want to run a command or script that outputs information to a file and sends this file via email or possibly FTP. +Or, you might want to view it in a text editor or print it out later. + +Once you have created the code (script, fragment, or a single command) that creates the output you need, you can use several techniques to send that output to a file. + +## The alternative methods + +There are (at least) four ways to get output to a file. +You can use any or all of: + +* `*-Content` cmdlets +* `Out-File` cmdlet +* Redirection operators +* .NET classes + +Writing this reminds me of my friends in Portugal who tell me there are 1000 ways to cook bacalao (cod). +Then they whisper: plus the way my mother taught me. +If there are more techniques for file output, I expect to see them in the comments to this article. 😃 + +## Using the `*-Content` cmdlets + +There are four `*-Content` cmdlets: + +* `Add-Content` - appends content to a file. +* `Clear-Content` - removes all content of a file. +* `Get-Content` - retrieves the content of a file. +* `Set-Content` - writes new content which replaces the content in a file. + +The two cmdlets you use to send command or script output to a file are `Set-Content` and `Add-Content`. +Both cmdlets convert the objects you pass in the pipeline to strings, and then output these strings to the specified file. +A very important point here - if you pass either cmdlet a non-string object, these cmdlets use each object's **ToString()** method to convert the object to a string before outputting it to the file. +For example: + +```powershell-console +PS> Get-Process -Name pwsh | Set-Content -Path C:\\Foo\\AAA.txt +PS> Get-Content -Path C:\\Foo\\AAA.txt +System.Diagnostics.Process (pwsh) +System.Diagnostics.Process (pwsh) +System.Diagnostics.Process (pwsh) +System.Diagnostics.Process (pwsh) +System.Diagnostics.Process (pwsh) +``` + +In many cases, this conversion does not produce what you expect (or want). +In this example, PowerShell found the 5 pwsh.exe processes, converted each to a string using **ToString()**, and outputs those strings to the file. +When you use **ToString** .Net's default implementation prints out the object's type name, like this: + +```powershell-console +PS> $Foo = [System.Object]::new() +PS> $Foo.ToString() +System.Object +``` + +The **System.Diagnostics.Process** class's implementation of the **ToString()** method is only marginally richer. +The **ToString()** method for this class outputs the object's type name and includes the process name as you see above. +But it is far short of the richer output you see when you use `Get-Process` from the console. + +The `*-Content` cmdlets are useful when you are building up a report programmatically. +For example, you could create a string, then add to it repeatedly in a script, finally outputting the report string to a file. +You can see the basic approach to building up a report in [this script that creates a Hyper-V VM summary report](https://github.com/doctordns/Wiley20/blob/master/10%20-%20Reporting/10.8%20-%20Creating%20a%20Hyper-V%20Status%20Report.ps1). + +You can improve the output from `Set-Content` by using `Out-String`, like this: + +```powershell-console +PS> # Get Powershell processes, convert to string, then output to a file +PS> Get-Process -Name pwsh | + Out-String | + Set-Content .\\Process.txt +PS> # View the file +PS> Get-Content .\\Process.txt + + NPM(K) PM(M) WS(M) CPU(s) Id SI ProcessName + ------ ----- ----- ------ -- -- ----------- + 70 56.65 109.05 13.19 2876 1 pwsh + 87 100.72 161.84 4.69 31252 1 pwsh + 63 54.40 93.90 22.27 31500 1 pwsh + 145 295.50 355.05 465.28 38132 1 pwsh + 64 52.82 95.29 52.95 38436 1 pwsh +``` + +Now that is looking a lot more like what I suspect you wanted! +But there is an easier way. + +## Using `Out-File` + +The `Out-File` cmdlet sends output to a file. +The cmdlet, however, uses PowerShell's formatting system to write to the file rather than using **ToString()**. +Using this cmdlet means Powershell sends the file the same display representation that you see from the console. + +Using `Out-File` looks like this: + +```powershell-console +PS> # Get Powershell processes and output to a file +PS> Get-Process -Name pwsh | Out-File -Path C:\\Foo\\pwsh.txt +PS> Get-Content -Path C:\\Foo\\pwsh.txt + + NPM(K) PM(M) WS(M) CPU(s) Id SI ProcessName + ------ ----- ----- ------ -- -- ----------- + 72 57.62 109.93 13.41 2876 1 pwsh + 92 136.95 202.20 5.44 31252 1 pwsh + 63 54.40 93.90 22.30 31500 1 pwsh + 145 295.49 355.05 465.80 38132 1 pwsh + 64 52.88 95.32 52.98 38436 1 pwsh + +``` + +The `Out-File` cmdlet gives you control over the output that PowerShell composes and sends to the file. +You can use the `-Encoding` parameter to tell PowerShell how to encode the output. +By default, PowerShell 7 uses the [UTF-8](https://en.wikipedia.org/wiki/UTF-8) encoding, but you can choose others should you need to. + +If you output very wide tables, you can use the `-Width` parameter to adjust the output's width. +In PowerShell 7, you can specify a value of up to 1024, enabling very wide tables. +Although the documentation does not specify any maximum upper value, formatting is erratic if you specify a width greater than 1025 characters. + +## The Redirection Operators + +There are two PowerShell operators you can use to redirect output: `>` and `>>`. +The `>` operator is equivalent to `Out-File` while `>>` is equivalent to `Out-File -Append`. +The redirection operators have other uses like redirecting error or verbose output streams. +You can read more about the [redirection operator(s) in the online help](https://docs.microsoft.com/powershell/module/microsoft.powershell.core/about/about_redirection). + +## Using .NET Classes + +There are several .NET classes you can leverage to produce output to a file. +C# developers have to use these classes since C# does not have PowerShell's formatting engine. +There are three classes, depending on your use case, that you might use: + +* [BinaryWriter](https://docs.microsoft.com/dotnet/api/system.io.binarywriter) - Writes primitive types in binary to a stream. +* [StreamWriter](https://docs.microsoft.com/dotnet/api/system.io.streamwriter) - writes characters to a stream in a particular encoding. +* [StringWriter](https://docs.microsoft.com/dotnet/api/system.io.stringwriter) - writes information to a string. With this class, Powershell stores the string information in a [StringBuilder](https://docs.microsoft.com/dotnet/api/system.text.stringbuilder) object. + +Of these three, the class you are most likely to use to send output to a file is the **StreamWriter** class. +Like this: + +```powershell +# Get the directories in C:\\ +$Dirs = Get-ChildItem -Path C:\\ -Directory +# Open a stream writer +$File = 'C:\\Foo\\Dirs.txt' +$Stream = [System.IO.StreamWriter]::new($File) +# Write the folder names for these folders to the file +foreach($Dir in $Dirs) { + $Stream.WriteLine($Dir.FullName) +} +# Close the stream +$Stream.Close() + ``` + +You can use `Get-Content` to view the generated content, like this: + +```powershell-console +PS> Get-Content -Path c:\\Foo\\Dirs.txt +C:\\AUDIT +C:\\Boot +C:\\Foo +C:\\inetpub +C:\\jea +C:\\NVIDIA +C:\\PerfLogs +C:\\Program Files +C:\\Program Files (x86) +C:\\PSDailyBuild +C:\\ReskitApp +C:\\Temp +C:\\Users +C:\\WINDOWS +``` + +For most PowerShell-using IT Pros, using the classes in the `System.IO` namespace is useful in two situations. +The first case is where you are doing a quick and dirty translation of a complex C# fragment to PowerShell. +The stream writer example above is based on the C# sample in the [SteamWriter's documentation page](https://docs.microsoft.com/dotnet/api/system.io.streamwriter). +In some cases, it might be easier to translate the code to PowerShell than to recode it to use cmdlets. +The second use case is where you are writing very large amounts of data to the file. +There is a limit on how big a .NET string object can be, restricting your report-writing. +If you are writing reports of tens of millions of lines of output (e.g. in an IoT scenario), writing one line at a time may be a way to avoid out of memory issues. +I doubt many IT Pros encounter such issues, but it's always a good idea to know there are alternatives where you need them. + +## Summary + +You have many options over how you send output to a file. +Each method has different use cases, as I mentioned above. +In most cases, I prefer using `Out-File`. +Using `Set-Content` is useful to set the initial contents of a file, for example, if you create a new script file based on a standard corporate template. +From the console, doing stuff quick/dirty, using the redirection operators can be useful alternatives. +Using the `System.IO` classes is another way to perform output and useful for very large output sets. +So lots of options - and I would not be surprised to find more methods I'd not considered! + +## Tip of the Hat + +I based this article on one written for the earlier Scripting GUys blog [How Can I Save Output to a Text File?](https://devblogs.microsoft.com/scripting/how-can-i-save-output-to-a-text-file/). +I am not sure who the author was. diff --git a/Posts/2021/06/media/tfl-edgestart/tfl-edgestgart.png b/Posts/2021/06/media/tfl-edgestart/tfl-edgestgart.png new file mode 100644 index 00000000..96e6d401 Binary files /dev/null and b/Posts/2021/06/media/tfl-edgestart/tfl-edgestgart.png differ diff --git a/Posts/2021/06/tfl-edgestart.md b/Posts/2021/06/tfl-edgestart.md new file mode 100644 index 00000000..d943a995 --- /dev/null +++ b/Posts/2021/06/tfl-edgestart.md @@ -0,0 +1,97 @@ +--- +post_title: How to Change the Start Page for the Edge Browser +username: tfl@psp.co.uk +Categories: PowerShell +tags: PowerShell, Edge +Summary: How to configure start page for the MS Edge web browser. +--- +**Q:** How can I change the Edge startup page? + +**A:** You can change the start page easily using PowerShell. + +## Edge and It's Start Page + +I am basing this article on the latest incarnation of the Edge browser, aka Edge Chromium. +The settings in this article seem to work fine on the latest versions of Windows 10 and Server 2022. +Other browsers can have different approaches to startup page. +And as ever - E&OE! + +When the Edge browser starts up, it displays a startup page based on Bing by default. +For many, this is fine - they can browse the contents and then navigate where they want. +But in some circumstances, you may wish to change this default. +And fortunately, this is straightforward to achieve. + +An easy way to set the startup page for yourself is to configure two registry value entries on the local machine. +The first is the **RestoreOnStartup** value entry to the registry key `HKCU:\\Software\\Policies\\Microsoft\\Edge`. +This value entry is a **REG_DWORD**. +By setting this entry with a value of **4**, you tell Edge to use the URL or URLs you specify when it starts up rather than the default home page. + +The second value entry (or entries) is within the key `HKCU:\\Software\\Policies\\Microsoft\\Edge\\RestoreOnStartupURLs`. +This value entry (or entries) contains the URL (or URLS) you want Edge to open at startup. +In most cases, you would setup a single URL under this key, but you can set more to have Edge bring up multiple pages at startup. + +Each registry value entry must have a unique entry name and contain the value of a URL you want Edge to restore at startup. +The value entry name doesn't seem to matter, so a value of **1** is fine. +If you want a second URL, then add a second value entry with a name of **2** (and the value of the second URL). + +Creating and setting these keys and key values as shown below enables the current user's settings. +If you are sharing the host with multiple users and want all users to have the same start page, you can set these entries in `HKCU:\\Software\\Policies` instead. + +## Configuring Edge Chromium Home Page + +You have choices as to what page of pages Edge opens when it starts. +You could choose a corporate Intranet landing page or perhaps a SharePoint project site. +You could also the startup page to a different search engine home page, such as Duck Duck Go. +As noted above, you can configure multiple startup pages should that be useful. + +With PowerShell, you use the `New-Item` cmdlet to create the registry keys. +It also makes sense to test whether the keys exist before trying to create them (and generating an error). +You set the value entries using the `Set-ItemProperty` cmdlet. + +The following code snippet sets the default home page for the current user for Edge Chromium to [DudkDuckGo](https://duckduckgo.com/). + +```powershell +# Ensure Edge key exists +$EdgeHome = 'HKCU:\\Software\\Policies\\Microsoft\\Edge' +If ( -Not (Test-Path $EdgeHome)) { + New-Item -Path $EdgeHome | Out-Null +} +# Set RestoreOnStartup value entry +$IPHT = @{ + Path = $EdgeHome + Name = 'RestoreOnStartup' + Value = 4 + Type = 'DWORD' +} +Set-ItemProperty @IPHT -verbose +# Create Startup URL's registry key +$EdgeSUURL = "$EdgeHome\\RestoreOnStartupURLs" +If ( -Not (Test-Path $EdgeSUURL)) { + New-Item -Path $EdgeSUURL | Out-Null +} +# Create a single URL startup page +$HOMEURL = 'https://duckduckgo.com' +Set-ItemProperty -Path $EdgeSUURL -Name '1' -Value $HomeURL +``` + +The next time you start Edge, you should see this: + +![Running Edge with new startup page](./media/tfl-edgestart/tfl-edgestgart.png) + + +## There are other ways + +As ever with PowerShell, there are other ways you could change start page for Edge. +You can [use the browser itself as described in a Microsoft suuport article](https://support.microsoft.com/en-us/microsoft-edge/change-your-browser-home-page-a531e1b8-ed54-d057-0262-cc5983a065c6). +As ever, you coulkd set the registry settings using WMI. +And, last but not least, it's [straightforward to set the browser's start page via group policy](https://admx.help/?Category=Windows_10_2016&Policy=Microsoft.Policies.MicrosoftEdge::HomePages). + +## Summary + +It is easy to change the start page for the Edge web browser +you just have to set a fewe registry keys and job done. + +## Tip of the Hat + +I based this article on one written for the earlier Scripting Guys blog [How Can I Change the Internet Explorer Home Page?](https://devblogs.microsoft.com/scripting/how-can-i-change-the-internet-explorer-home-page/). +I don't know the author. diff --git a/Posts/2021/06/tfl-rename-nic.md b/Posts/2021/06/tfl-rename-nic.md new file mode 100644 index 00000000..68d62516 --- /dev/null +++ b/Posts/2021/06/tfl-rename-nic.md @@ -0,0 +1,102 @@ +--- +post_title: How to rename a NIC +username: tfl@psp.co.uk +Categories: PowerShell +tags: PowerShell, network, NIC +Summary: How to rename a NIC using PowerShell +--- +**Q:** Is there a simple way to rename a NIC, especially inside a Hyper-V VM? + +**A:** You can change the name of any Windows NIC using PowerShell - whether the NIC is in a physical host or a Hyper-V VM. + + +## NICS and NIC names + +One thing that can quickly become confusing when using Hyper-V with multiple VMs and VM Switches is how fast the network adapters seem to proliferate. +You start with a few wired Ethernet Adapters on the host. +Then you install Hyper-V and create a VM farm with loads of virtual NICs. +Before you know it, you have a dozen adapters inside the VM host and an unclear set of adapters in the VM. + +To discover the NICs in a host or a VM, you use the `Get-NetAdapter` cmdlet. +Which looks like this inside a Hyper-V VM: + +```powershell-Console +PS> Get-NetAdapter + +Name InterfaceDescription ifIndex Status MacAddress LinkSpeed +---- -------------------- ------- ------ ---------- --------- +Ethernet Microsoft Hyper-V Network Adapter 22 Up 00-15-5D-01-2A-91 10 Gbps +Ethernet 2 Microsoft Hyper-V Network Adapter #2 14 Up 00-15-5D-01-2A-92 10 Gbps +Ethernet 3 Microsoft Hyper-V Network Adapter #3 15 Up 00-15-5D-01-2A-92 10 Gbps +Ethernet 4 Microsoft Hyper-V Network Adapter #4 16 Up 00-15-5D-01-2A-92 10 Gbps +Local Area Connection TAP-Windows Adapter V9 12 Disconnected 00-FF-B6-68-E1-5D 1 Gbps +``` + +Once you add a few NICs to a VM, each connected to a separate switch, telling them apart can be challenging. +To help you with subsequent maintenance, it can be good to rename the adapter and change the description. +Renaming a VM's NICs is a good habit to get into - and is straightforward to achieve. +Before renaming anything, ensure you determine the purpose for each NIC in your VM. +Once you work out what use each NIC plays in your VM farm, you can use the `Rename-NetAdapter` cmdlet to rename the NIC. + +There are two ways you could use `Rename-NetAdapter` to rename one of our NICs, like this: + +```powershell +# Using a 'Get-Rename' pattern +Get-NetAdapter -InterfaceIndex 22 | Rename-NetAdapter -NewName 'Reskit Management' +# Just Using Rename-NetAdapter +Rename-NetAdapter -InterfaceIndex 22 -NewName 'Reskit Management' +``` + +I rarely, if ever, rename a NIC using a production script since it is usually a one-off operation. +For that reason, I prefer to use the first method. +I can first use `Get-NetAdapter` on its own to ensure I'm getting the right adapter. +Then, I can hit `Up-Arrow`, and pipe the previous command to `Rename-NetAdapter` and specify a new name for the NIC. + +## Admin rights required + +There is just one small snag with using `Rename-NetAdapter` - you have to run it in an elevated console. +If, as I often do, forget to run PowerShell as an administrator, you would see the following when attempting to rename the NIC: + +```powershell-console +PS> Get-NetAdapter -InterfaceIndex 22 | Rename-NetAdapter -NewName 'Sales iSCSI VLAN' +Rename-NetAdapter: Access is denied. +``` + +Although it might have been nice to tell you to run the command in an elevated PowerShell console, the error message should be clear enough. +And, interestingly, this fact is not currently mentioned in the help text. + +Assuming that you are an administrator with the rights to change a NIC's name, you can open a new elevated PowerShell session and try the command again. +If you are using PSReadLine, when you start up the new console (as an Administrator), the command should be in PSReadLine's command cache. +And that means, once the new console is up and available, you can access that earlier command by hitting up-arrow and then hitting return. + +When you do, you see this: + +```powershell-console +PS> Get-NetAdapter -InterfaceIndex 16 | Rename-NetAdapter -NewName 'Sales iSCSI VLAN' +PS> Get-NetAdapter -InterfaceIndex 16 + +Name InterfaceDescription ifIndex Status MacAddress LinkSpeed +---- -------------------- ------- ------ ---------- --------- +Sales iSCSI VLAN Microsoft Hyper-V Network Adapter 16 Up 00-15-5D-01-2A-91 10 Gbps +``` + +In this example: `Rename-NetAdapter` did change the name of the adapter but produced no console output. +You use `Get-NetAdapter` to view the new name. + +## There are other ways + +As ever with PowerShell, there are other ways you could change the name of a NIC. +One more old-fashioned way would be to use the `netsh.exe` program. +And then there is WMI - you can use the `Set-CimInstance` to perform the name change. +And I look forward to comments suggesting other ways to change a NIC's name. + +## Summary + +It is easy to change a network adapter's name. +Unfortunately, the Rename-NetAdapter does not allow you to change the interface description. +You need to run the `Rename-NetAdapter` in an elevated console - if you don't, you get an Access Denied error. + +## Tip of the Hat + +I based this article on one written for the earlier Scripting Guys blog [Renaming Network Adapters by Using PowerShell](https://devblogs.microsoft.com/scripting/renaming-network-adapters-by-using-powershell/). +The author of that article was Ed Wilson. diff --git a/Posts/2021/07/tfl-regkey.md b/Posts/2021/07/tfl-regkey.md new file mode 100644 index 00000000..289b742b --- /dev/null +++ b/Posts/2021/07/tfl-regkey.md @@ -0,0 +1,166 @@ +--- +post_title: How to Update or Add a Registry Key Value with PowerShell +username: tfl@psp.co.uk +Categories: PowerShell +tags: PowerShell, registry, provider +Summary: How you can update or add registry keys or registry key value entries. +--- +**Q:** I am having a problem trying to update the registry. +I am using the New-ItemProperty cmdlet, but it fails if the registry key does not exist. +I added the –Force parameter, but it still does not create the registry key. +The error message says that it cannot find the path because it does not exist. +Is there something I am not doing? I include my script so you can see what is going on. Help me, please? + +**A:** Let's look at how you can use PowerShell to add or update any registry key value. + +## The Registry + +Before answering the query, let me cover some of the background basics. +You probably already know this but I start with a look at the Registry and how PowerShell providers relate to the query. +I hope this is not _too_ basic! + +In Windows the Registry is a database of configurations information used by Windows and Windows applications. +The registry is critical to the operation of Windows - I learned that long ago (and got practice reinstalling Windows NT). +Using the registry editor can be dangerous, so be careful! + +The registry is a set of hierarchical keys - a registry key can have zero, or more sub-keys, and so on. +Each key or sub-key can have zero or more value entries. +Each value entry has a data type and a data value. +Any registry key can have values of any data type. +The registry allows you to create any key and to put pretty much any kind of data into a value entry. + +The registry is implemented in Windows as a set of registry hives. +A hive is a logical group of keys, sub-keys, and values in the registry. +Each hive has a set of supporting files that Windows loads into memory when the operating system starts up or a user logs in. +For more details about registry hives see [the Registry Hives on-line help text](https://docs.microsoft.com/windows/win32/sysinfo/registry-hives). + +Ever since Windows NT 3.1, it is easy to edit the registry using the built in registry editor - **regedit.exe**. +Windows NT also had the **reg.exe** command that allowed you to manage the registry programatically and you can still usew it today. +You can also use the WMI to access WMI, as shown in this excerpt from [Richard Siddaway's book **PowerShell and WMI**](Https://livebook.manning.com/book/powershell-and-wmi/chapter-7/). + +For IT Pros using PowerShell, the Windows PowerShell team, created a very simple way through the use of the Registry provider which is the focus of this article. + +## Providers and the Registry Provider + +Windows contains a number of data stores that are critical to the operation of Windows and Windows applications. +These data stores include the registry, as well as the file store, the certificate store, and more. +The developers of PowerShell, when faced with the challenge of enabling IT Pros to access all this information had two main options. + +The first option was to create a huge number unique cmdlets for each data store +This would be a lot of work and would be almost certain to introduce inconsistencies. +The second option was to use an intermediate layer, the provider, which converted the data store into something resembling the file store. +With the provider you use the same command(s) to get access the registry, access files and folders, etc. + +To discover the providers on your system, you use the `Get-PSProvider` cmdlet like this: + +```powershell-console +PS> Get-PSProvider + +Name Capabilities Drives +---- ------------ ------ +Registry ShouldProcess {HKLM, HKCU} +Alias ShouldProcess {Alias} +Environment ShouldProcess {Env} +FileSystem Filter, ShouldProcess, Credentials {C, D, H, I, M, N, Temp, db… +Function ShouldProcess {Function} +Variable ShouldProcess {Variable} +Certificate ShouldProcess {Cert} +``` + +## Provider Drives + +With a provider, you can create a drive that allows access to part of one of the provider-based data stores. +For the filestore provider, PowerShell provides you with provider drives pointing to the Windows volumes in your system, such as **C:**, **D:**, etc. +You can also create a provider drive called `DB:` that points to `D:\\Dropbox` by using the `New-PSDrive` cmdlet. +You can persist the drive name by adding the statement to your profile should this be useful. + +With the registry provider, PowerShell provides you with two built-in drives: `HKLM:` and `HKCU:`. +The **HKLM:** drive exposes the local machine registry hive - which you (and Windows) use for system wide settings. +You use the **HKCU:** drive to access the current user's registry hive. + +You can discover the provider based drives by using the `Get-PSProvider` cmdlet, like this: + +```powershell-console +PS> Get-PSDrive + +Name Used (GB) Free (GB) Provider Root +---- --------- --------- -------- ---- +Alias Alias +C 262.51 714.58 FileSystem C:\\ +Cert Certificate \\ +D 1312.83 596.76 FileSystem D:\\ +db 1312.83 596.76 FileSystem D:\\DropBox +docs 1312.83 596.76 FileSystem D:\\Dropbox\\PACKT… +Env Environment +F FileSystem F:\\ +Function Function +G 2.68 56.79 FileSystem G:\\ +gd 3169.18 556.84 FileSystem M:\\gd +H 2860.16 865.85 FileSystem H:\\ +HKCU Registry HKEY_CURRENT_USER +HKLM Registry HKEY_LOCAL_MACHINE.. +``` +Some Windows features come with additional providers, such as the the **ActiveDirectory** RSAT module. +This feature includes an AD provider: + +```powershell-console +PS> Import-Module -Name ActiveDirectory +PS> Get-PSProvider -Name ActiveDirectory + +Name Capabilities Drives +---- ------------ ------ +ActiveDirectory Include, Exclude, Filter, ShouldProcess, Credentials {AD} +``` + +## Registry Value Entries + +As I mentioned above, a registry key can contain value entries. +You can think of each value entry as an attribute of a registry key. +You use the `*-ItemProperty` cmdlets to manage individual registry values. +But how does this relate to the question? +Let's begin by looking at the script in question: + +```powershell-console +$RegistryPath = 'HKCU:\\Software\\CommunityBlog\\Scripts' +$Name = 'Version' +$Value = '42' +New-ItemProperty -Path $RegistryPath -Name $Name -Value $Value -PropertyType DWORD -Force + +New-ItemProperty: Cannot find path 'HKCU:\\Software\\CommunityBlog\\Scripts' because it does not exist. +``` + +The script used the `New-ItemProperty` to create a **Version** value entry to a specific key. +This script, however, fails since the registry key, specified in `$RegistryPath` variable does not exist. + +A better approach is to test the registry key path first, creating it if needed, then setting the value entry, like this: + +```powershell +# Set variables to indicate value and key to set +$RegistryPath = 'HKCU:\\Software\\CommunityBlog\\Scripts' +$Name = 'Version' +$Value = '42' +# Create the key if it does not exist +If (-NOT (Test-Path $RegistryPath)) { + New-Item -Path $RegistryPath -Force | Out-Null +} +# Now set the value +New-ItemProperty -Path $RegistryPath -Name $Name -Value $Value -PropertyType DWORD -Force +``` +## A small word of warning + +Playing with the registry can be dangerous. +This is true when using both the Registry Editor and the PowerShell commands. +Be careful! + +## Summary + +It is easy to change add registry keys and values. +You can use the `New-Item` cmdlet to create any key in any registry hive. +Once you create the key, you can use `New-ItemProperty` to set a registry value entry. + + + +## Tip of the Hat + +I based this article on one written for the earlier Scripting Guys blog [Update or Add Registry Key Value with PowerShell](https://devblogs.microsoft.com/scripting/update-or-add-registry-key-value-with-powershell/). +It was written by Ed Wilson. diff --git a/Posts/2021/08/How Can I Be Notified Any Time a Service Goes Down.md b/Posts/2021/08/How Can I Be Notified Any Time a Service Goes Down.md new file mode 100644 index 00000000..4c6d389f --- /dev/null +++ b/Posts/2021/08/How Can I Be Notified Any Time a Service Goes Down.md @@ -0,0 +1,187 @@ +--- +post_title: How can I be notified any time a service goes down? +username: farisnt@gmail.com +Categories: PowerShell +tags: PowerShell, WMI, Events +Summary: Using Powershell to create temporary event monitoring using WMI + +--- + +Q: How can I be notified any time a service goes down? + +A: The short quick answer to utilizing WMI and PowerShell 7. + +You use PowerShell to create temporary event monitoring using WMI. +Then WMI monitors any service changes and generates an alert once it detects a change. + +## Basic Requirement + +To achieve this, you need Windows PowerShell 5.1 and above. + +This post uses the latest version of PowerShell 7. +So if you are don't yet have PowerShell 7, see the Microsoft documentation on how to [Install PowerShell 7 on Windows](https://docs.microsoft.com/powershell/scripting/install/installing-powershell-core-on-windows). + +Also, make sure that PowerShell is running as administrator. + +## Finding the Required Class + +Before going into details, you need to find the required class to monitor. +To get a list of all available classes, use the following code. + +```powershell-console +PS> Get-CimClass -Namespace root\\cimv2 + + NameSpace: ROOT/CIMV2 + +CimClassName CimClassMethods CimClassProperties +------------ --------------- ------------------ +__SystemClass {} {} +__thisNAMESPACE {} {SECURITY_DESCRIPTOR} +__Provider {} {Name} +__Win32Provider {} {Name, ClientLoadableCLSID, CLSID, Concurrency…} +__ProviderRegistration {} {provider} +__EventProviderRegistration {} {provider, EventQueryList} +__ObjectProviderRegistration {} {provider, InteractionType, QuerySupportLevels, SupportsBatch… +__ClassProviderRegistration {} {provider, InteractionType, QuerySupportLevels, SupportsBatch… +__InstanceProviderRegistration {} {provider, InteractionType, QuerySupportLevels, SupportsBatch… +__MethodProviderRegistration {} {provider} +__PropertyProviderRegistration {} {provider, SupportsGet, SupportsPut} +__EventConsumerProviderRegistration {} {provider, ConsumerClassNames} +``` + +The returned result represents all the available classes in the namespace. +For this tutorial, the focus is on Windows services, which is represented by **Win32_Service**. + +To Enumerate the **Win32_Services** WMI class and get all the available services using PowerShell run the following code. + +```powershell-console +PS> Get-CimInstance -Namespace root\\CIMV2 -ClassName win32_service + +ProcessId Name StartMode State Status ExitCode +--------- ---- --------- ----- ------ -------- +3784 AdobeARMservice Auto Running OK 0 +3792 AdobeUpdateService Auto Running OK 0 +3800 AGMService Auto Running OK 0 +3824 AGSService Auto Running OK 0 +0 AJRouter Manual Stopped OK 1077 +0 ALG Manual Stopped OK 1077 +0 AppIDSvc Manual Stopped OK 1077 +6708 Appinfo Manual Running OK 0 +21444 AppMgmt Manual Running OK 0 +0 AppReadiness Manual Stopped OK 1077 +0 AppVClient Disabled Stopped OK 1077 +0 AppXSvc Manual Stopped OK 0 +0 AssignedAccessManagerSvc Manual Stopped OK 1077 +``` + +For now, you can use PowerShell to find the required class, and enumerate it to get the available services. +Let's go deeper now and start creating the WMI Event Subscription. + +There are three steps you need to follow to create a temporary WMI Event Subscription: + +- Create a WMI query language query. +- Register this query. +- Obtain any events generated. + +## Creating WMI Query + +WMI has many special classes that you can use to detect changes to other WMI classes. +For example, you can use the **CIM_InstModification** class to monitor the targeted class, in this case **Win32_Service** + +You have to create a WMI query using [WMI Query Language](https://docs.microsoft.com/windows/win32/wmisdk/wql-sql-for-wmi). + +The WQL syntex structure looks like this: + +``` +Select * from <WMI System Class> within <Number of Seconds> where TargetInstance ISA <WMI Class name> +``` + +Let apply the same to **Win32_Serivce**. Start by creating a PowerShell variable, in our case, you construct the query as follows: + +```powershell-console +$query = "Select * from CIM_InstModification within 10 where TargetInstance ISA 'Win32_Service'" +``` + +A full explanation for the WQL query is available in [Your Goto Guide for Working with Windows WMI Events and PowerShell](https://adamtheautomator.com/your-goto-guide-for-working-with-windows-wmi-events-and-powershell/). + +## Registering The Query + +We have the WQL query, let's move to the next and register the query to the WMI events by using the [Register-CimIndicationEvent](https://docs.microsoft.com/powershell/module/cimcmdlets/register-cimindicationevent). +The `Register-CimIndicationEvent` is used to subscribe to events generated from the system. +And in our case, it subscribes to events generated from the `$query`. + +```powershell-console +Register-CimIndicationEvent -Namespace 'ROOT\\CIMv2' -Query $query -SourceIdentifier 'WindowsServices' -MessageData 'Service Status Change' +``` + +To confirm the successful registration, type the following cmdlet `Get-EventSubscriber`, the output looks like the following + +```powershell-console +PS> Get-EventSubscriber + +SubscriptionId : 1 +SourceObject : Microsoft.Management.Infrastructure.CimCmdlets.CimIndicationWatcher +EventName : CimIndicationArrived +SourceIdentifier : **WindowsServices** +Action : +HandlerDelegate : +SupportEvent : False +ForwardEvent : False +``` + +And that's all that we need, simple as that. + +## Reading the events + +Now the event is registered and active. +Next, create an event, and by that, I mean stopping or starting a service. + +Try *Windows Update* service (wuauserv), run the following cmdlet to see the status of the bits service. + +```powershell-console +PS> Get-Service wuauserv + +Status Name DisplayName +------ ---- ----------- +Running wuauserv Windows Update +``` + +So the service is running, let stop it by typing the following + +```powershell +PS> Stop-Service wuauserv +``` + +To see the newly created events, type `Get-Event` +Look at the **MessageData**, it's the same message used in the `Register-CimIndicationEvent`. + +```powershell-console +PS> $EventVariable=Get-Event +PS> $EventVariable + +ComputerName : +RunspaceId : 91c6b6fb-cda9-4b15-983f-d7af1f639358 +EventIdentifier : 1 +Sender : Microsoft.Management.Infrastructure.CimCmdlets.CimIndicationWatcher +SourceEventArgs : Microsoft.Management.Infrastructure.CimCmdlets.CimIndicationEventExceptionEventArgs +SourceArgs : {Microsoft.Management.Infrastructure.CimCmdlets.CimIndicationWatcher, + Microsoft.Management.Infrastructure.CimCmdlets.CimIndicationEventExceptionEventArgs} +SourceIdentifier : WindowsServices +TimeGenerated : 30-Jul-21 12:08:06 AM +MessageData : Service Status Change +``` + +To find the current state of this event + +```powershell-console +PS> $EventVariable.SourceEventArgs.NewEvent.PreviousInstance + +ProcessId Name StartMode State Status ExitCode +--------- ---- --------- ----- ------ -------- +16508 wuauserv Manual Running OK 0 +``` + +This WMI monitoring remains active as long as the PowerShell console. +It creates such a temporary job which runs in the background to monitor the services class. +You can also end this process by rebooting the computer. +Hope you learned something new today. diff --git a/Posts/2021/09/Understanding Get-ACL and AD Drive Output.md b/Posts/2021/09/Understanding Get-ACL and AD Drive Output.md new file mode 100644 index 00000000..f9591d08 --- /dev/null +++ b/Posts/2021/09/Understanding Get-ACL and AD Drive Output.md @@ -0,0 +1,257 @@ +--- +post_title: Understanding Get-ACL and AD Drive Output +username: farisnt@gmail.com +Categories: PowerShell +tags: PowerShell, Active Directory, ACL +Summary: Understanding Get-ACL and AD Drive Output +--- + +## Understanding `Get-ACL` and AD: Drive Output + +Understanding Active Directory ACL using PowerShell can be a bit tricky. +There are no out-of-the-box cmdlets with ActiveDirectory PowerShell module to help in settings the permission quickly. +While there are no cmdlets, you can nevertheless manage AD permissions using the AD PowerShell drive. + +In this post, I will try to simplify Active Directory ACL and how to read the result easily, so let's start. + +## Prerequisites + +To follow along with this article, you need the following: + +- PowerShell 7. x or Windows PowerShell 5.1 +- A user account that is member of Domain Admin AD Group. +- Windows Server 2012, 2016, 2019 or 2022 with Active Directory Domain Service role installed and participating in a domain. + +The domain name used for this tutorial is **Contoso.com**. + +## Reading Active Directory Permission using `Get-ACL` + +Reading Active Directory permission using `Get-ACL` doesn't require a long line of code. +However, we are reading from AD and not the FileSystem provider. So we use the `AD:` drive. This drive is automatically loaded when you load the ActiveDirectory module. + +[alert type="note" heading="Note"]To import Active Directory Module, use the `Import-Module ActiveDirectory`.[/alert] + +When you query for an object to get its ACL, you need to search based on **Distinguished Name**. +Use the following statement to get the ACL for the **MyOrgOU** organization unit in the **Contoso.com**. + +```powershell-console +PS> (Get-Acl -Path "AD:OU=MyOrgOU,DC=Contoso,DC=com").Access + +ActiveDirectoryRights : CreateChild, DeleteChild +InheritanceType : None +ObjectType : bf967aba-0de6-11d0-a285-00aa003049e2 +InheritedObjectType : 00000000-0000-0000-0000-000000000000 +ObjectFlags : ObjectAceTypePresent +AccessControlType : Allow +IdentityReference : BUILTIN\\Account Operators +IsInherited : False +InheritanceFlags : None +PropagationFlags : None +. +. +Output Trimmed to make the result clear. +``` + +As the output shows, there are multiple properties, but some are not obvious in terms of their usage. +You can tell from the first view is that there is a **CreateChild** and **DeleteChild** permission assigned to the **BUILTIN\\Account Operators**. +But on which object (User, Computer...etc) and which AD attribute? +Let's take another example. +You created user **User1** and assigned permission on **MyOrgOU** organizational unit by another administrator. +Using the `Get-ACL` cmdlet, return the following results. + +```powershell-console +PS> (Get-Acl -Path "AD:OU=MyOrgOU,DC=Contoso,DC=Com").Access | Where-Object {$_.IdentityReference -Like "Contoso\\User1"} + +ActiveDirectoryRights : WriteProperty +InheritanceType : Descendents +ObjectType : 28630ebf-41d5-11d1-a9c1-0000f80367c1 +InheritedObjectType : bf967aba-0de6-11d0-a285-00aa003049e2 +ObjectFlags : ObjectAceTypePresent, InheritedObjectAceTypePresent +AccessControlType : Allow +IdentityReference : Contoso\\User1 +IsInherited : False +InheritanceFlags : ContainerInherit +PropagationFlags : InheritOnly +``` + +Same as the first case, the question is what **Contoso\\User1** actually has permission to do? + +## Understanding the Get-ACL and AD Drive Output + +To make things easier, let's start by understanding each property of the output and what that property does.. + +### Understanding the ActiveDirectoryRights Property + +- **ActiveDirectoryRights**: The **ActiveDirectoryRights** refer to what rights are assigned to the AD object; +usually, this is readable, like **WriteProperty**, **DeleteProperty**. +But this is not always the case. +The **ActiveDirectoryRights** can also hold **ExtendedRights**, **Generic**, and other values. + +[alert type="note" heading="Note"]You can read more about **ActiveDirectoryRights** on the [ActiveDirectoryRights](https://docs.microsoft.com/en-us/dotnet/api/system.directoryservices.activedirectoryrights?view=net-5.0) page.[/alert] + +- The **ExtendedRight** flag means permission is set to a very specific AD object attribute, such as setting the write _pwdLastSet_ to a AD user object attribute. + ![Customized property permission = ExtendedRights in ActiveDirectoryRights](./media/getaclad/ExtADPermission.png) +- **Generic**: Some generic permission values include + - **GenericAll**: Equivalent to Full Control, so the user with *GenericAll* has full control permission on the object. + - **GenericRead**: Can read all object properties and permission and list content if its a container or OU. + - **GenericWrite**: Can write to all object's properties and permission. + +## Understanding the InheritanceType Property + +The **InheritanceType** shows the scope of the applied permission and defines which AD objects the ACE should + be applied to. +You can see the **InheritanceType** in the ACL GUI in the Advance Security Permission Window. + +![Applies to](./media/getaclad/InheritanceType_In_GUI.png) +The **InheritanceType** can hold one of the following values: + - **None**: The permission is applied to the object where the permission is set. +The **Applies to** is set to **This Object Only** + - **All**: The permission is applied to the object where the permission is set and all the child items in the tree. +The **Applies to** is set to **This object and all descendant objects**. + - **Descendents**: The permission is applied to child items only but not to the object where the permission is set. +The **Applies to** is set to **All descendant objects**. +Think of **OU1** and a child **OU2**. +**User1** has the following permission on **OU1** + +```powershell-console + PS> (Get-Acl -Path "AD:OU=OU1,DC=Contoso,DC=Com").Access | Where-Object {($_.IdentityReference -Like "*User1*")} + + ActiveDirectoryRights : GenericAll + InheritanceType : Descendents + ObjectType : 00000000-0000-0000-0000-000000000000 + InheritedObjectType : 00000000-0000-0000-0000-000000000000 + ObjectFlags : None + AccessControlType : Allow + IdentityReference : Contoso\\User1 + IsInherited : False + InheritanceFlags : ContainerInherit + PropagationFlags : InheritOnly +``` + +User1 can perform an action on **OU2** and **OU1** child items but not on the **OU1** object. + +- **Children**: set permissions on the direct child only, not the on the object itself neither any descendants object of its children. +You get this value when the **Applies to** set to any value other than **This object only** or **This object and all descendant objects** and at the same time the **Only apply this permission to objects and/or containers within this container** check box is selected. +- **SelfAndChildren**: Set permissions on the object itself where the permission is placed and the direct child only. +You get this value when the **Applies to** set to value **This object and all descendant objects** and at the same time the **Only apply this permission to objects and/or containers within this container** check box is selected. + +### Understanding the ObjectType Property + +The **ObjectType** is represented by a GUID value, even though this is one of the most important values that should be clear. +The output of `Get-ACL` makes it complex to understand. +The **ObjectType** is the object attribute. For example, in the following output, **User1** is **Allowed** to **WriteProperty** to object **28630ebf-41d5-11d1-a9c1-0000f80367c1**. + +```powershell-console +PS> (Get-Acl -Path "AD:OU=MyOrgOU,DC=Contoso,DC=Com").Access | where-Object {$_.IdentityReference -Like "Contoso\\user1"} + +ActiveDirectoryRights : WriteProperty +InheritanceType : Descendents +ObjectType : 28630ebf-41d5-11d1-a9c1-0000f80367c1 +InheritedObjectType : bf967aba-0de6-11d0-a285-00aa003049e2 +ObjectFlags : ObjectAceTypePresent, InheritedObjectAceTypePresent +AccessControlType : Allow +IdentityReference : Contoso\\User1 +IsInherited : False +InheritanceFlags : ContainerInherit +PropagationFlags : InheritOnly +``` + +To resolve this GUID to a name we need to see where these GUIDs are stored. + +To get the list of *ObjectType* names, run the following PowerShell code + +```powershell +$ObjectTypeGUID = @{} + + +$GetADObjectParameter=@{ + SearchBase=(Get-ADRootDSE).SchemaNamingContext + LDAPFilter='(SchemaIDGUID=*)' + Properties=@("Name", "SchemaIDGUID") +} + +$SchGUID=Get-ADObject @GetADObjectParameter + Foreach ($SchemaItem in $SchGUID){ + $ObjectTypeGUID.Add([GUID]$SchemaItem.SchemaIDGUID,$SchemaItem.Name) +} + +$ADObjExtPar=@{ + SearchBase="CN=Extended-Rights,$((Get-ADRootDSE).ConfigurationNamingContext)" + LDAPFilter='(ObjectClass=ControlAccessRight)' + Properties=@("Name", "RightsGUID") +} + +$SchExtGUID=Get-ADObject @ADObjExtPar + ForEach($SchExtItem in $SchExtGUID){ + $ObjectTypeGUID.Add([GUID]$SchExtItem.RightsGUID,$SchExtItem.Name) +} + +$ObjectTypeGUID | Format-Table -AutoSize +``` + +Take a look at the output, you see that there is the ObjectType GUID and the Name what an ObjectType GUID. + +```powershell-console +Name Value +---- ----- +a8032e74-30ef-4ff5-affc-0fc217783fec NisNetgroupTriple +5b7eae84-7e67-4d56-8fca-9cee24d19a65 ms-Exch-Policy-Tag-Link +5245803a-ca6a-11d0-afff-0000f80367c1 NTFRS-Replica-Set +52458038-ca6a-11d0-afff-0000f80367c1 Admin-Property-Pages +52458039-ca6a-11d0-afff-0000f80367c1 Shell-Property-Pages +296de070-b098-11d2-aa06-00c04f8eedd8 ms-Exch-Server2-Page-Size +203d2f32-b099-11d2-aa06-00c04f8eedd8 ms-Exch-Source-BH-Address +b8d47e4e-4b78-11d3-aa75-00c04f8eedd8 ms-Exch-Security-Password +b8fe00a9-8e59-4d4d-8939-85b79de4d8cf ms-Exch-Provisioning-Flags +4d7ea1cd-43a0-4255-9bb0-12f17be23ffb ms-Exch-ESE-Param-Replay-Background-Database-Maintenance-Delay +2dbb448a-5d85-4144-a9a5-2fc724e194a8 ms-Exch-Auto-Discover-Flags +e85e1204-3434-41ad-9b56-e2901228fff0 MS-DRM-Identity-Certificate +28630ebf-41d5-11d1-a9c1-0000f80367c1 Lockout-Time +28630ebe-41d5-11d1-a9c1-0000f80367c1 Is-Defunct +28630ebd-41d5-11d1-a9c1-0000f80367c1 Tree-Name +28630ebc-41d5-11d1-a9c1-0000f80367c1 Legacy-Exchange-DN +2d485eee-45e1-4902-add1-5630d25d13c2 ms-Exch-UM-Enabled-Flags +28630eba-41d5-11d1-a9c1-0000f80367c1 Service-DNS-Name-Type +``` + +So if you have an **ObjectType** GUID, you can search through the hashtable, and as you can see below, the **ObjectType** GUID is **Lockout-Time** + +```powershell-console +PS> $ObjectTypeGUID[[GUID]'28630ebf-41d5-11d1-a9c1-0000f80367c1'] +Lockout-Time +``` + +[alert type="note" heading="Note"]If you want to know more about Hashtables, read this great post on [Everything you wanted to know about hashtables](https://docs.microsoft.com/en-us/powershell/scripting/learn/deep-dives/everything-about-hashtable?view=powershell-7.1)[/alert] + +From the previous example and after understanding the **ObjectType** value, we know that **Contoso\\User1** is **Allowed* to *WrtiteProperty** to **Lockout-Time** property + +[alert type="note" heading="Note"]One side note, if the value of the **ObjectType** was 00000000-0000-0000-0000-000000000000 this means that the user is allowed//denied to all properties, not a specific one.[/alert] + +## Understanding InheritedObjectType Property + +The **InheritedObjectType** GUID represents the targeted object such as a **Computer**, **User**, **Contact**...etc., For example, when delegating the Helpdesk to unlock user's account, the **InheritedObjectType** value is **User** which the GUID of **bf967aba-0de6-11d0-a285-00aa003049e2** represents. +You can see the value in the GUI from here + +![User Object](./media/getaclad/InheritedObjectType.png) + +## Understanding IsInherited, InheritanceFlags, and PropagationFlags + +The Inheritance and how the object is inherited to the child object defined by three properties: + +- **IsInherited**: Object is inherited from a parent object, and the possible values are true or false. +- **InheritanceFlags**: Two values this flag can have are: + - **None**: The ACE won't inherit to the child items. It's only applied to the object it's set to. + - **ContainerInherit**: The ACE is inherited to the child items. +- **[PropagationFlags](https://docs.microsoft.com/en-us/dotnet/api/system.security.accesscontrol.propagationflags?view=net-5.0)**: control how the ACE is propagated to the child items, and the possible values are: + - **None**: Specifies that no inheritance flags are set. + - **InheritOnly**: The ACE is applied to the child items only, not the object where the ACE is set. + - **NoPropagateInherit**: The ACE is applied on the object where the ACE is set not propagated to any child. +You can see this value when the *Only Apply this permission to objects and/or containers within this container* is selected. + + ![Apply to This Container only](./media/getaclad/applythiscontainer.png) + +Keep in mind that the **PropagationFlags** are significant only if inheritance flags are present. + +## Conslucsion +As you can see, the `Get-ACL` cmdlet is a powerful cmdlet, and it works in the `AD:` drive as the FileSystem drive, but with some challenges. +In this post, I try to simplify the `Get-ACL` output result and explain the GUIDs and what these GUID mean, as it helps in better understanding the permission structure through PowerShell. diff --git a/Posts/2021/09/media/getaclad/ExtADPermission.png b/Posts/2021/09/media/getaclad/ExtADPermission.png new file mode 100644 index 00000000..e56ce3cf Binary files /dev/null and b/Posts/2021/09/media/getaclad/ExtADPermission.png differ diff --git a/Posts/2021/09/media/getaclad/InheritanceType_In_GUI.png b/Posts/2021/09/media/getaclad/InheritanceType_In_GUI.png new file mode 100644 index 00000000..7ffa2969 Binary files /dev/null and b/Posts/2021/09/media/getaclad/InheritanceType_In_GUI.png differ diff --git a/Posts/2021/09/media/getaclad/InheritedObjectType.png b/Posts/2021/09/media/getaclad/InheritedObjectType.png new file mode 100644 index 00000000..f1bf1cbe Binary files /dev/null and b/Posts/2021/09/media/getaclad/InheritedObjectType.png differ diff --git a/Posts/2021/09/media/getaclad/applythiscontainer.png b/Posts/2021/09/media/getaclad/applythiscontainer.png new file mode 100644 index 00000000..03e9abf2 Binary files /dev/null and b/Posts/2021/09/media/getaclad/applythiscontainer.png differ diff --git a/Posts/2021/09/my-crescendo-journey.md b/Posts/2021/09/my-crescendo-journey.md new file mode 100644 index 00000000..35a5a706 --- /dev/null +++ b/Posts/2021/09/my-crescendo-journey.md @@ -0,0 +1,166 @@ +--- +Categories: PowerShell +post_title: My Crescendo journey +Summary: How I stopped worrying and learned to create a module +tags: Crescendo, module +username: sewhee@microsoft.com +--- +In a recent PowerShell Users Group meeting I was thinking that it might be good to talk about the +new [Crescendo][blog1] module and how to use it. I was going to ask [Jason Helmick][jason] if he +would do a presentation for us. Then, in an unrelated conversation, someone mentioned using +`vssadmin.exe` for some project. This got me thinking: `vssadmin` is a perfect candidate for a +Crescendo module and maybe I should just learn it and do the presentation myself. + +## What is Crescendo? + +Crescendo is an experimental module developed by [Jim Truher][jim], one of the main developers of +PowerShell. Crescendo provides a framework to rapidly develop PowerShell cmdlets that wrap native +commands, regardless of platform. The goal of a Crescendo-based module is to create PowerShell +cmdlets that use a native command-line tool, but unlike the tool, return PowerShell objects instead +of plain text. + +## How I got started + +When I first heard about Crescendo, I thought: + +> _So what. I've written wrapper modules like this before. How is this going to help me?_ + +But I knew there must be more to it for Jim to invest this much time and effort into it, and I +wanted something to present at the user group meeting. + +So, I started by reading the blog posts about Crescendo and looking at some examples in the +[repository][repo]. + +### How Crescendo works + +The to create a module using the Crescendo framework you have to create two main components: + +- A JSON configuration file that describes the cmdlets you want +- Output handler functions that parse the output from the native command and return objects + +Initially, the parsing code had to be embedded in the JSON file, which made writing and formatting +the code very difficult. But, in the [Preview 3][blog3] release, Jim added the ability to create +your output handler code in a function or a script file, making it much easier to manage. + +Alright! Writing the PowerShell functions is something I am more comfortable with, so that was my +next step. + +## Writing the output parser functions + +To create the parser functions I had to know what the output looked like for all of the possible +command combinations of `vssadmin.exe`. I looked at the help provided by `vssadmin` and captured the +output for each subcommand in a separate file. I used these output files to design and implement a +parsing function for each subcommand. + +Now, on to the configuration file. + +## Creating the JSON configuration + +For this I used the example from the [blog post][blog3] as a template. I also looked at the +`Get-InstalledPackage` example from the [Preview 2][blog2] blog post to see how the native commands +were referenced. For my first cmdlet I started with this JSON configuration: + +```json +{ + "$schema": "https://aka.ms/Crescendo/Schema.json", + "Commands": [ + { + "Verb": "Get", + "Noun": "VssProvider", + "OriginalName": "$env:Windir/system32/vssadmin.exe", + "OriginalCommandElements": [ + "list", + "providers" + ], + "OutputHandlers": [ + { + "ParameterSetName": "Default", + "HandlerType": "Function", + "Handler": "ParseProvider" + } + ], + } + ] +} +``` + +The `ParseProvider` function is one of the functions that I had written to parse the output. I +repeated this pattern to create a new cmdlet for each of the `vssadmin` subcommands. + +Notice that the first line of the JSON references a schema file. This file comes with the Crescendo +module. I used Visual Studio Code (VS Code) to do all my development. With this schema file, VS Code +provides IntelliSense for the JSON, making it easy to know which values are required and the type of +information needed. + +Eventually, I added properties to the configuration for full help with descriptions and examples. +And I defined parameter sets for the `vssadmin` commands that supported parameters. + +## Creating the new module + +Crescendo, itself, is a module. It contains cmdlets that help you create your configuration and then +uses that configuration to create the module containing your cmdlets. Once I was happy with the +configuration file, I used the `Export-CrescendoModule` cmdlet to create my module. + +```powershell +Export-CrescendoModule -ConfigurationFile .\vssadmin.crescendo.config.json -ModuleName VssAdmin.psm1 +``` + +Crescendo created two new files: + +- The module code file `VssAdmin.psm1` +- The module manifest file `VssAdmin.psd1` + +These are the only two files that need to be installed. The `VssAdmin.psm1` file contains all the +cmdlets that Crescendo generated from the configuration and the **Output Handler** functions I +wrote to parse the output into objects. + +The end result was a well-structured, fully documented module. + +I still have one cmdlet left to create and I want to add administrative elevation since `vssadmin` +requires it. But I am happy with the results I have so far. + +## Conclusion + +After reading all of this you might still be asking "how is this any easier than just writing the +module myself?" + +That is a fair question. But here are the conclusions I came to as I went through this process. + +- The whole process, starting from nothing, researching both Crescendo and `vssadmin`, writing the + code, creating the configuration, and generating the module took me about 4 hours. I thought that + was pretty fast. +- Crescendo lets you separate the logic code (your parsing functions) from the cmdlet definition and + parameter handling code. I found it easier to describe the cmdlets and their parameters in the + JSON file rather than having to write that code myself. +- Crescendo handles things like **CommonParameters** and `SupportsShouldProcess` for you. You don't + have to write that support code in the cmdlets. +- The configuration file also makes it easy to add help to your cmdlets. You don't have to remember + the comment-based help keywords and structure. +- Separating the declarative code (the JSON configuration) from the logical code (your parsers) + makes it easier to add functionality to your module if the native command-line tool is updated. + +Take a few minutes to read the Crescendo blog posts. Then go and look at the VssAdmin module I +created. I have included the link to it below. Examine the `vssadmin.crescendo.config.json` file to +see how I defined the cmdlets and the parameter sets. The `vssadmin.exe resize shadowstorage` +command has a `/MaxSize=` parameter that can take 3 different types of values. Look at the +definition of the `Resize-VssShadowStorage` cmdlet to see how I handled that. + +## Links to resources + +- The blog posts + - [Announcing Crescendo Preview 1][blog1] + - [Announcing Crescendo Preview 2][blog2] + - [Announcing Crescendo Preview 3][blog3] +- My [VssAdmin][vssadmin] module +- The [Crescendo repository][repo] on GitHub +- The [Microsoft.PowerShell.Crescendo][gallery] module on the PowerShell Gallery + +<!-- link references --> +[blog1]: https://devblogs.microsoft.com/powershell/announcing-powershell-crescendo-preview-1/ +[blog2]: https://devblogs.microsoft.com/powershell/announcing-powershell-crescendo-preview-2/ +[blog3]: https://devblogs.microsoft.com/powershell/announcing-powershell-crescendo-preview-3/ +[jim]: https://devblogs.microsoft.com/powershell/author/jimtrumicrosoft-com/ +[jason]: https://devblogs.microsoft.com/powershell/author/jahelmic/ +[repo]: https://github.com/PowerShell/Crescendo +[vssadmin]: https://github.com/sdwheeler/modules/vssadmin +[gallery]: https://www.powershellgallery.com/packages/Microsoft.PowerShell.Crescendo diff --git a/Posts/2021/09/tfl-profile.md b/Posts/2021/09/tfl-profile.md new file mode 100644 index 00000000..9dfd7da8 --- /dev/null +++ b/Posts/2021/09/tfl-profile.md @@ -0,0 +1,126 @@ +--- +post_title: How to Make Use Of PowerShell Profile Files +username: tfl@psp.co.uk +Categories: PowerShell +tags: PowerShell, profile +Summary: Using Profile files with PowerShell 7 +--- + +**Q:** I would like to personalize the way that PowerShell works. +I have heard that I can use a thing called a profile to do this, but when I try to find information about profiles, I come up blank. There is no `New-Profile` cmdlet, so I do not see how to create such a thing. Can you help me, please? + +**A:** Profiles are a powerful part of PowerShell and allow you to customize PowerShell for your environment. +They are easy to create and support a range of deployment scenarios. + +## What is a Profile? + +Before explaining the profile, let's first examine the PowerShell host. +A PowerShell host is a program that hosts PowerShell to allow you to use it. +Common PowerShell hosts include the Windows PowerShell console, the Windows PowerShell ISE, the PowerShell 7 console, and VS Code. +Each host supports the use of profile files. + +A profile is a PowerShell script file that a PowerShell host loads and executes automatically every time you start that PowerShell host. +The script is, in effect, dot-sourced, so any variables, functions, and the like that you define in a profile script remain available in the PowerShell session, which is incredibly handy. +I use profiles to create PowerShell drives, various useful variables, and a few useful (for me!) functions. + +Each PowerShell host has 4 separate profile files as follows: + +* This host, this user +* This host, all users +* All hosts, this user +* All hosts, all users + +Why so many, you might ask. +Because having these four profile files allows you numerous deployment opportunities. +You could, for example, have one profile that defines corporate aliases or standard PS drives for every PowerShell host and user on a machine. +You could have 'this host' profiles that define host-specific customizations that could differ depending on the PowerShell host. +For example, in my profile file for VS code, I use `Set-PSReadLineOption` to set token colours depending on which color theme I am using. +Like so many things in PowerShell, the PowerShell team engineered profiles for every scenario you might come across in deploying PowerShell and PowerShell hosts. + +In practice, the "this host, this user" profile is the one you most commonly use, but having all four allows considerable deployment flexibility. +You have options! + +## Where do I find them? + +Another frequently asked question is: where are these files and how are they named? +It turns out, like many things PowerShell, you can find the answer to the question inside PowerShell itself. +In this case, inside a PowerShell automatic variable, `$PROFILE`. + +Automatic variables in PowerShell, are variables created by PowerShell itself and are available for use. +These variables are created by PowerShell when you start the host. +For more details on automatic variables see the [automatic variable help text](https://docs.microsoft.com/powershell/module/microsoft.powershell.core/about/about_automatic_variables). + +## The `$PROFILE` variable + +The `$PROFILE` variable is an automatic variable that PowerShell creates within each session during startup. +This variable has both a **ToString()** method and four additional note properties that tell you where _this_ host finds its profile files. + +To determine the location and fill script name for the four PowerShell scripts, you can do something like this: + +```powershell-console +PS> # what host? +PS> $host.Name +ConsoleHost +PS> # Where are the profiles? +PS> $PROFILE | Get-Member -MemberType NoteProperty + TypeName: System.String +Name MemberType Definition +---- ---------- ---------- +AllUsersAllHosts NoteProperty string AllUsersAllHosts=C:\\Program Files\\PowerShell\\7\\profile.ps1 +AllUsersCurrentHost NoteProperty string AllUsersCurrentHost=C:\\Program Files\\PowerShell\\7\\Microsoft.PowerShell_profile.ps1 +CurrentUserAllHosts NoteProperty string CurrentUserAllHosts=C:\\Users\doctordns\\Documents\\PowerShell\\profile.ps1 +CurrentUserCurrentHost NoteProperty string CurrentUserCurrentHost=C:\\Users\\doctordns\\Documents\\PowerShell\\Microsoft.PowerShell_profile.ps1 + +PS> # What does the $PROFILE variable itself contain? +PS> $PROFILE +C:\\Users\\doctordns\\Documents\\PowerShell\\Microsoft.PowerShell_profile.ps1 +``` + +This example is from a Windows 10 client running PowerShell 7 inside VS Code. +In the example, you can see that the `$PROFILE` variable contains four note properties that contain the location of each profile +Also, you can see that the `$PROFILE` variable's value is the name of the **CurrentUserCurrentHost** profile. +For simplicity you can run `Notepad $Profile` to bring up the profile file inside Notepad (or use VS Code!) + +## What can you do in a profile script? + +You can pretty much do anything you want in profile file to create the environment that works best for you. +I find the profile useful for creating variables and short aliases, PS Drives, and more as you can see below. +As an example of what you can do in a profile, and to get you started, I have published two sample profile files to GitHub: + +* A [profile for the PowerShell 7 console](https://github.com/doctordns/PACKT-PS7/blob/master/scripts/goodies/Microsoft.PowerShell_Profile.ps1) +* A [profile for VSCode](https://github.com/doctordns/PACKT-PS7/blob/master/scripts/goodies/Microsoft.VSCode_profile.ps1) + +These samples do a lot of useful things, including: + +* Over-ride some default parameter values +* Update the Format enumeration limit +* Set the 'home' directory to a non-standard location +* Create personal aliases +* Create a PowerShell credential object + +These are all things that make the environment customized to your liking. +I use some personal aliases as alternatives to standard aliases - if only to save typing. +Creating personal variables or updating automatic variables can be useful. + +While creating a credential object can be useful, it is arguable whether it is a good thing. +In this case, the credential is for a set of VMs I used in my [most recent PowerShell book](https://smile.amazon.co.uk/Windows-Server-Automation-PowerShell-Cookbook-ebook/dp/B0977JDL7K/ref=sr_1_1?dchild=1&keywords=Windows+Server+Automation+with+PowerShell+Cookbook+-+Fourth+Edition&qid=1624277697&s=books&sr=1-1) to illustrate using PowerShell in an Enterprise. +As they are all local VMs and are only for testing, creating a much used credential object is useful. + +## Be Careful + +It is easy to get carried away with profile files. +At one point in the PowerShell 3.0 days, my profile file was over 700 lines long. +I'd just chucked all these cool things I'd found on the Internet (and never used them again) +As a result, starting PowerShell or the ISE took some time. +It is so easy to see some cool bits of code and then add it to your profile. +I suggest you look carefully at each profile on a regular basis and trim it when possible. + +## Summary + +Profile are PowerShell scripts you can use to customize your PowerShell environment. +There are 4 profile files for each host as you can see by examining the `$Profile` automatic variable. + +## Tip of the Hat + +I based this article on one written for the earlier Scripting Guys blog [How Can I Use Profiles With Windows PowerShell](https://devblogs.microsoft.com/scripting/hey-scripting-guy-how-can-i-use-profiles-with-windows-powershell/). +It was written by Ed Wilson. diff --git a/Posts/2021/10/Expresso.png b/Posts/2021/10/Expresso.png new file mode 100644 index 00000000..77db83d2 Binary files /dev/null and b/Posts/2021/10/Expresso.png differ diff --git a/Posts/2021/10/crescendo-configuration.md b/Posts/2021/10/crescendo-configuration.md new file mode 100644 index 00000000..925f3036 --- /dev/null +++ b/Posts/2021/10/crescendo-configuration.md @@ -0,0 +1,351 @@ +--- +Categories: PowerShell +post_title: A closer look at the Crescendo configuration +Summary: In this post I take a closer look at a cmdlet definition in the Crescendo configuration file. +featured_image: https://devblogs.microsoft.com/powershell-community/wp-content/uploads/sites/69/2021/09/Crescendo.png +tags: Crescendo, cmdlet, json, configuration +username: sewhee@microsoft.com +--- +In my [previous post][3], I looked at the details of a Crescendo output handler from my +[VssAdmin][7] module. In this post, I explain the details of a cmdlet definition in the Crescendo +JSON configuration file. + +## The purpose of the configuration + +The structure for the interface of a cmdlet is a reasonably predictable thing. + +- The cmdlet uses a standard _Verb-Noun_ format +- The cmdlets take input using sets of parameters +- Cmdlets that make changes to the system support `-Confirm` and `-WhatIf` parameters + +The pattern of the script code to support these fits a template. + +The more difficult part of the cmdlet is in the code that does the work. Crescendo separates the +functional code (the output handler) from the cmdlet interface code. The Crescendo configuration +file defines the interfaces of cmdlets that you want Crescendo to create. + +The Crescendo configuration file is a JSON file containing an array of cmdlet definitions. JSON +provides an expressive, structured syntax for defining the properties of objects. But so does, +PowerShell. So why use JSON and not a PowerShell data (PSD1) file? The answer is simple: schema! +Unlike PowerShell's PSD1 files, JSON supports a schema. Having a schema ensures that the syntax of +your definition is correct. And with tools like Visual Studio Code (VS Code), the schema provides +IntelliSense, making it easier to author. + +## Defining a cmdlet interface + +The structure of a cmdlet definition can be divided into three property categories in the JSON +file: + +- Required properties + - **Verb** + - **Noun** + - **OriginalName** + - **OriginalCommandElements[]** + - **OutputHandlers[]** +- As-required properties + - **DefaultParameterSetName** + - **Parameters[]** +- Nice-to-have properties + - "Help" properties like **Description**, **Usage**, and **Examples[]** + +You might notice that defining **Parameters** is optional. This is not uncommon. In my VssAdmin +module, the cmdlets `Get-VssProvider`, `Get-VssVolume`, and `Get-VssWriter` do not have parameters. +These simple cmdlets don't require any input to return the requested information. + +Let's take a closer look at a simple cmdlet definition. + +- The **Verb** and **Noun** properties form the name of the cmdlet. +- The **OriginalName** property contains the path to the native command that the cmdlet runs to get + the output. +- The **OriginalCommandElements** is an array of strings that are passed to the native command as + parameters. A typical CLI like `vssadmin` has its own set of commands that perform different + actions. Those commands may have additional parameters. In this example, the + `vssadmin list providers` command has no additional parameters. +- The **OutputHandlers** property is an array containing one or more handler definitions. The + handlers receive the output of the native command and return an object containing the data parsed + from the output. + - The **HandlerType** can be `Inline`, `Function`, or `Script`. In this example I use `Function`. + - The **Handler** is the name of the Script or Function to be called, or the actual PowerShell + code to run if the type is `Inline`. + +```json +{ + "$schema": "https://aka.ms/Crescendo/Schema.json", + "Commands": [ + { + "Verb": "Get", + "Noun": "VssProvider", + "OriginalName": "$env:Windir/system32/vssadmin.exe", + "OriginalCommandElements": [ + "list", + "providers" + ], + "Description": "List registered volume shadow copy providers", + "Usage": { + "Synopsis": "List registered volume shadow copy providers" + }, + "Examples": [ + { + "Command": "Get-VssProvider", + "Description": "Get a list of VSS Providers", + "OriginalCommand": "vssadmin list providers" + } + ], + "OutputHandlers": [ + { + "ParameterSetName": "Default", + "HandlerType": "Function", + "Handler": "ParseProvider" + } + ] + } + ] +} +``` + +The remaining properties -- **Description**, **Usage**, and **Examples** -- are optional. +Crescendo uses these values to create the comment-based help for the cmdlet when it creates the +module. + +## Defining parameters and parameter sets + +Some of the `vssadmin` commands have optional parameters that can be used in various combinations. +For example: + +- `vssadmin List Shadows [/For=ForVolumeSpec] [/Shadow=ShadowId|/Set=ShadowSetId]` - 3 optional + parameters in 2 parameter sets +- `vssadmin List ShadowStorage [/For=ForVolumeSpec|/On=OnVolumeSpec]` - 2 parameter sets with 1 + optional parameter each + +Let's take a look at the help for `vssadmin Resize ShadowStorage`. + +``` +vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool +(C) Copyright 2001-2013 Microsoft Corp. + +Resize ShadowStorage /For=ForVolumeSpec /On=OnVolumeSpec /MaxSize=MaxSizeSpec + - Resizes the maximum size for a shadow copy storage association between + ForVolumeSpec and OnVolumeSpec. Resizing the storage association may cause shadow + copies to disappear. As certain shadow copies are deleted, the shadow copy storage + space will then shrink. If MaxSizeSpec is set to the value UNBOUNDED, the shadow copy + storage space will be unlimited. MaxSizeSpec can be specified in bytes or as a + percentage of the ForVolumeSpec storage volume. For byte level specification, + MaxSizeSpec must be 320MB or greater and accepts the following suffixes: KB, MB, GB, TB, + PB and EB. Also, B, K, M, G, T, P, and E are acceptable suffixes. To specify MaxSizeSpec + as percentage, use the % character as the suffix to the numeric value. If a suffix is not + supplied, MaxSizeSpec is in bytes. + + Example Usage: vssadmin Resize ShadowStorage /For=C: /On=D: /MaxSize=900MB + vssadmin Resize ShadowStorage /For=C: /On=D: /MaxSize=UNBOUNDED + vssadmin Resize ShadowStorage /For=C: /On=C: /MaxSize=20% +``` + +The `vssadmin Resize ShadowStorage` command has three required parameters, but the third parameter +`/MaxSize` can take three different types of input. In PowerShell, we prefer fixed types for +parameter values. We can solve this by creating three different parameters, each used in a +different parameter set. + +The following JSON defines the `Resize-VssShadowStorage` cmdlet. The definition starts with the +required properties and some help information. This definition also has **SupportsShouldProcess** +set to `true`. With this property, Crescendo adds the `[SupportsShouldProcess()]` attribute to the +cmdlet, which automatically adds the `-WhatIf` and `-Confirm` parameters. + +The interesting part starts in the parameter definitions. + +```json + { + "Verb": "Resize", + "Noun": "VssShadowStorage", + "OriginalName": "c:/windows/system32/vssadmin.exe", + "OriginalCommandElements": [ + "Resize", + "ShadowStorage" + ], + "Description": "Resizes the maximum size for a shadow copy storage association between ForVolumeSpec and OnVolumeSpec. Resizing the storage association may cause shadow copies to disappear. As certain shadow copies are deleted, the shadow copy storage space will then shrink.", + "Usage": { + "Synopsis": "Resize the maximum size of a shadow copy storage association." + }, + "Examples": [ + { + "Command": "Resize-VssShadowStorage -For C: -On C: -MaxSize 900MB", + "Description": "Set the new storage size to 900MB", + "OriginalCommand": "vssadmin Resize ShadowStorage /For=C: /On=C: /MaxSize=900MB" + }, + { + "Command": "Resize-VssShadowStorage -For C: -On C: -MaxPercent '20%'", + "Description": "Set the new storage size to 20% of the OnVolume size", + "OriginalCommand": "vssadmin Resize ShadowStorage /For=C: /On=C: /MaxSize=20%" + }, + { + "Command": "Resize-VssShadowStorage -For C: -On C: -Unbounded", + "Description": "Set the new storage size to unlimited", + "OriginalCommand": "vssadmin Resize ShadowStorage /For=C: /On=C: /MaxSize=UNBOUNDED" + } + ], + "SupportsShouldProcess": true, + "DefaultParameterSetName": "ByMaxSize", + "Parameters": [ + { + "OriginalName": "/For=", + "Name": "For", + "ParameterType": "string", + "ParameterSetName": [ "ByMaxSize", "ByMaxPercent", "ByMaxUnbound" ], + "NoGap": true, + "Description": "Provide a volume name like 'C:'" + }, + { + "OriginalName": "/On=", + "Name": "On", + "ParameterType": "string", + "ParameterSetName": [ "ByMaxSize", "ByMaxPercent", "ByMaxUnbound" ], + "Mandatory": true, + "NoGap": true, + "Description": "Provide a volume name like 'C:'" + }, + { + "OriginalName": "/MaxSize=", + "Name": "MaxSize", + "ParameterType": "Int64", + "ParameterSetName": [ "ByMaxSize" ], + "AdditionalParameterAttributes": [ + "[ValidateScript({$_ -ge 320MB})]" + ], + "Mandatory": true, + "NoGap": true, + "Description": "New maximum size in bytes. Must be 320MB or more." + }, + { + "OriginalName": "/MaxSize=", + "Name": "MaxPercent", + "ParameterType": "string", + "ParameterSetName": [ "ByMaxPercent" ], + "AdditionalParameterAttributes": [ + "[ValidatePattern('[0-9]+%')]" + ], + "Mandatory": true, + "NoGap": true, + "Description": "A percentage string like '20%'." + }, + { + "OriginalName": "/MaxSize=UNBOUNDED", + "Name": "Unbounded", + "ParameterType": "switch", + "ParameterSetName": [ "ByMaxUnbound" ], + "Mandatory": true, + "Description": "Sets the maximum size to UNBOUNDED." + } + ], + "OutputHandlers": [ + { + "ParameterSetName": "ByMaxSize", + "HandlerType": "Function", + "Handler": "ParseResizeShadowStorage" + }, + { + "ParameterSetName": "ByMaxPercent", + "HandlerType": "Function", + "Handler": "ParseResizeShadowStorage" + }, + { + "ParameterSetName": "ByMaxUnbound", + "HandlerType": "Function", + "Handler": "ParseResizeShadowStorage" + } + ] + } +``` + +The parameters have the following properties: + +- **OriginalName** contains the original parameter used by the native command. Crescendo combines + the value passed into the cmdlet with the original parameter string. The resulting native + parameter is added to the original native command that gets executed by the cmdlet. +- **Name** is the name of the parameter for the PowerShell cmdlet you are defining. +- **ParameterType** is the type of the parameter for the cmdlet. +- **ParameterSetName** is an array of one or more parameter set names that the parameter belongs to. +- **AdditionalParameterAttributes** is an array of strings that contain any additional attribute you + want added to the parameter. You can use this to add parameter validation attributes. +- **NoGap** tell Crescendo not so use a space between the **OriginalName** parameter and the value + passed into the cmdlet. +- **Description** is the description of the parameter displayed by `Get-Help`. + +For this cmdlet, the first two parameters `-For` and `-On` are in all three parameter sets. The +remaining three parameters are unique to each parameter set. + +- The `-MaxSize` parameter accepts a 64-bit integer. That value is added to the `/MaxSize=` string + to form the native parameter. The parameter validation ensures that the value passed in is greater + than 320MB. +- The `-MaxPercent` parameter accepts a string containing a percentage value. That string is added + to the `/MaxSize=` string to form the native parameter. The parameter validation ensures that the + string represents a valid percentage. +- The `-Unbounded` switch parameter is used select a native parameter of `/MaxSize=UNBOUNDED`. + +## Defining the output handlers + +Since there are three parameters sets, I need to define an output handler for each set. You could +have a separate function for each set. In my case that was not necessary. The +`vssadmin Resize ShadowStorage` command does not have any output unless there is an error. Also, +since the command makes changes, I thought I should call `Get-VssShadowStorage` to show the new +settings. + +```powershell +function ParseResizeShadowStorage { + param( + [Parameter(Mandatory)] + $cmdResults + ) + $textBlocks = ($cmdResults | Out-String) -split "`r`n`r`n" + + if ($textBlocks[1] -like 'Error*') { + Write-Error $textBlocks[1] + } elseif ($textBlocks[1] -like 'Success*') { + Get-VssShadowStorage + } else { + $textBlocks[1] + } +} +``` + +## The final step + +Once the configuration file was complete, I used the `Export-CrescendoModule` cmdlet to create my +**VssAdmin** module. + +```powershell +Export-CrescendoModule -ConfigurationFile vssadmin.crescendo.config.json -ModuleName VssAdmin.psm1 +``` + +Crescendo created two new files: + +- The module code file - `VssAdmin.psm1` +- The module manifest file - `VssAdmin.psd1` + +These are the only two files that need to be installed. The `VssAdmin.psm1` file contains all the +cmdlets that Crescendo generated from the configuration and the Output Handler functions I wrote to +parse the output into objects. + +## Conclusion + +Crescendo separates the structural interface code required to create a cmdlet from the functional +code that extracts the data. The configuration file defines the cmdlet interfaces. The +`Export-CrescendoModule` cmdlet creates a new module containing the cmdlets defined in the +configuration (complete with the help text provided) and the output handler functions required by +the cmdlets. It also creates a proper module manifest, complete with exports for the new cmdlets. + +## Resources + +Posts in this series + +- [My Crescendo journey][1] + - [My VssAdmin module][7] +- [Converting string output to objects][2] +- [A closer look at a Crescendo Output Handler][3] +- A closer look at a Crescendo configuration file - this post + +References + +<!-- link reference --> +[1]: https://devblogs.microsoft.com/powershell-community/my-crescendo-journey/ +[2]: https://devblogs.microsoft.com/powershell-community/converting-string-output-to-objects/ +[3]: https://devblogs.microsoft.com/powershell-community/a-closer-look-at-the-parsing-code-of-a-crescendo-output-handler/ +[7]: https://github.com/sdwheeler/tools-by-sean/tree/master/modules/vssadmin diff --git a/Posts/2021/10/crescendo-output-handler.md b/Posts/2021/10/crescendo-output-handler.md new file mode 100644 index 00000000..c052b2f9 --- /dev/null +++ b/Posts/2021/10/crescendo-output-handler.md @@ -0,0 +1,148 @@ +--- +Categories: PowerShell +post_title: A closer look at the parsing code of a Crescendo output handler +Summary: In this post I take a close look at one of my output handler and talk about the different parsing methods I used. +tags: Crescendo, output handler, parsing +username: sewhee@microsoft.com +--- +In my [previous post][2], I showed you how to parse the output from the `netstat` command. The +output of `netstat` is not very complex. The goal of the post was to introduce some parsing +strategies that you can use to create a full Crescendo module. In this post, I explain the details +of a more complex parsing function that I created for my [VssAdmin module][7]. + +## Examining the parser for `Get-VssShadowStorage` + +The following screenshot shows the `ParseShadowStorage` function that is called by the +`Get-VssShadowStorage` cmdlet to handle the output from `vssadmin.exe`. The parsing process is +broken into four areas: + +- Collect the native command output into a single text blob then split it into blocks of text (lines + 96-100) +- Parse each line of a text block (lines 105-132) +- Collect the parsed data in a hashtable (lines 104, 119, and 129) +- Return the collected data as an object (line 133) + +![ParseShadowStorage function](crescendo-parser.png) + +### Collecting the native command output + +The output from `vssadmin.exe list shadowstorage` is passed into the output handler function as an +array of strings. Here is an example of the output: + +``` +vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool +(C) Copyright 2001-2013 Microsoft Corp. + +Shadow Copy Storage association + For volume: (C:)\\\\?\\Volume{67a44989-8413-4a7c-a616-79385dae8605}\\ + Shadow Copy Storage volume: (C:)\\\\?\\Volume{67a44989-8413-4a7c-a616-79385dae8605}\\ + Used Shadow Copy Storage space: 22.1 GB (4%) + Allocated Shadow Copy Storage space: 22.5 GB (4%) + Maximum Shadow Copy Storage space: 23.7 GB (5%) +``` + +All the `vssadmin.exe` commands followed this pattern. There is a 2-line header followed by one or +more groups of lines of data. Each group of text is separated by a blank line. So first, I wanted to +get the text split into the blocks separated by the blank line. + +Piping the `$cmdresults` parameter to `Out-String` turns that array of lines into one contiguous +blob of text. Then, I split that blob into blocks of text at the blank line using the +``-split "`r`n`r`n"`` operator. The `$textblocks` variable now contains 2 block of text. The first +block is the header and the second block is the data. Using the `for` loop on line 102 I start +proccessing the lines in the second text block. I can skip the first since is only contains the +header. + +### Parsing the lines of the textblocks + +On line 105, I split the text block into an array of lines so that I can process each line. Looking +at the example data above, I can see that there are two kinds of information: _volume_ and _space_ +information. + +Since both information sets have the same basic format, I only need one parser for each type of +information. Beginning on line 107, the `foreach` loop iterates over each line of the text block. On +line 108, the `switch` statement uses regular expression matching (regex) to determine whether the +line contains _volume_ or _space_ information. Lines 109-120 parse the volume information. Lines +121-130 parse the space information. + +In both cases, I want to collect the data in a key/value pair. The line is split into two parts +by the **"volume:"** or **"space:"** strings. + +The text of the first part is used to create the key name. + +- For volume information, I remove the spaces from the text to form the key names: **ForVolume** and + **ShadowCopyStorageVolume**. +- For space information, I split out the first word to form the key names: **UsedSpace**, + **AllocatedSpace**, and **MaximumSpace**. + +The second part contains the value data. Each information type contains two data values: + +- For volume information: the volume name and path + + In line 113, I use a [regex with named groups](#expresso) to isolate the volume name and path from + the string data. Those values are assigned to properties of the `$volinfo` object, which becomes + the value for the key/value pair. + +- For space information: the size and percent usage + + To get to the space data items, I split the string at the open parenthesis `(` and trim off the + closing parenthesis. The first part becomes the size value and the second part becomes the + percentage. Similar to the volume data, these values are assigned to properties of the `$space` + object, which becomes the value of the key/value pair. + +### Returning the object + +I now have a key/value pair ready to be added to a hashtable in line 119 or 129. Once all the lines +have been parsed, the hashtable is complete. Line 133 converts the hashtable to a **PSObject**, +which is returned to the `Get-VssShadowStorage` cmdlet function for output. + +## Conclusion + +I use this same strategy to create parsing functions for all of the `vssadmin` command outputs. + +- Collect the native command output as a single text blob +- Split the blob into separate text blocks +- Parse each line of a text block to extract the data +- Collecting the parsed data in a hashtable +- Return the collected data as an object + +I used several different methods to parse the strings into data: + +- String class methods (`Split()`, `Trim()`, `Replace()`) +- PowerShell Operators (`-split` and `-match`) +- Regular expressions (with `-match` and `switch`) + +Each native command has its own unique output. Crescendo does not help you analyze or parse that +output. You must take the time to collect the output, analyze the structure of the information, and +devise a strategy for extracting the data from the strings. + +Crescendo separates the structural code required to create a cmdlet from the functional code that +extracts the data. In my next post, I will take a close look at the JSON configuration file that +Crescendo uses to define the structural code. + +## Resources + +Posts in this series + +- [My Crescendo journey][1] + - [My VssAdmin module][7] +- [Converting string output to objects][2] +- A closer look at a Crescendo Output Handler - this post +- A closer look at a Crescendo configuration file - coming soon + +References + +- [about_Regular_Expressions][3] +- [about_Operators][4] +- [about_Switch][5] +- [Expresso by Ultrapico][6] - my favorite (and free) regular expression tool + + <a id='expresso'>![expresso](Expresso.png)</a> + +<!-- link reference --> +[1]: https://devblogs.microsoft.com/powershell-community/my-crescendo-journey/ +[2]: https://devblogs.microsoft.com/powershell-community/converting-string-output-to-objects/ +[3]: https://docs.microsoft.com/powershell/module/microsoft.powershell.core/about/about_regular_expressions +[4]: https://docs.microsoft.com/powershell/module/microsoft.powershell.core/about/about_operators +[5]: https://docs.microsoft.com/powershell/module/microsoft.powershell.core/about/about_switch +[6]: http://www.ultrapico.com/Expresso.htm +[7]: https://github.com/sdwheeler/tools-by-sean/tree/master/modules/vssadmin diff --git a/Posts/2021/10/crescendo-parser.png b/Posts/2021/10/crescendo-parser.png new file mode 100644 index 00000000..df3d94ba Binary files /dev/null and b/Posts/2021/10/crescendo-parser.png differ diff --git a/Posts/2021/10/netstat-output.png b/Posts/2021/10/netstat-output.png new file mode 100644 index 00000000..22541b32 Binary files /dev/null and b/Posts/2021/10/netstat-output.png differ diff --git a/Posts/2021/10/parsing-netstat.md b/Posts/2021/10/parsing-netstat.md new file mode 100644 index 00000000..4ca6fe9c --- /dev/null +++ b/Posts/2021/10/parsing-netstat.md @@ -0,0 +1,210 @@ +--- +Categories: PowerShell +post_title: Converting string output to objects +Summary: How to turn the text output from a native command like `netstat` into a PowerShell object. +tags: parsing strings, netstat +username: sewhee@microsoft.com +--- +In my [previous post][1], I talked about using Crescendo to create a PowerShell module for the +`vssadmin.exe` command in Windows. As I explained, you have to write Output Handler code that parses +the output of the command you are using. But if you never written a parser like this, where do you +start? + +In this post I show you how to parse the output from the `netstat` command. The output of `netstat` +is not very complex and it is basically the same on Windows and Linux systems. The goal here is to +talk about parsing strategies that you can use to create a full Crescendo module. + +## Step 1 - Capture the output + +To create a parser you have to capture the output so you can analyze it deeply enough to understand +the structure. Capturing the output is easy. + +```powershell +netstat > netstat-output.txt +``` + +## Step 2 - Analyze the output + +The goal of this analysis is to isolate the important data points. There are several question you +want to answer as you look at the captured output. + +- What are the individual data points being displayed? +- How is the data labeled? +- What information needs to be parsed and what can be ignored? +- What repeating patterns exist in the output? + - Look for delimiters and labels +- Does the data format change? What formats must be handled? + +![netstat output](netstat-output.png) + +Here are my observations about the output from `netstat`. + +1. There is only one set of header lines. The output is not divided in to multiple sections with + different headers. The column headers contain spaces in the column names making parsing more + difficult. +1. The output is presented as a table. The columns are labeled (Proto, Local Address, Foreign + Address, State). Each row of the table is formatted the same with spaces separating the columns. +1. The **Address** columns contain a mix of IP Addresses and Host names, both with ports. The ports + can be numeric or text. +1. The IP Addresses can be formatted as IPv4 or IPv6 addresses. The IPv6 addresses are enclosed in + brackets (`[]`). +1. There are no space character inside the data columns but there are colon characters. This makes + the space character a good delimiter, as opposed to the colon. + +Now I can start writing code for the parser. + +## Step 3 - Write the parser + +From my analysis, I can tell that I am really only interested in rows of data. I don't care about +the table header because it never changes. So I can just ignore it. The first row of data starts +after the table header. There are two ways to get data passed to your parsing function: + +- Streaming data on the pipeline + + If I am streaming data, my parser function must accept input from the pipeline and I must look for + the header line then start parsing the data on the next line. + +- Passing the entire output from the command as the value for a parameter + + If the data is passed in as a single object, then I can just skip to the first line of data to + begin parsing. This is the method I am going to use in this example. + +Getting the output of `netstat` into a variable is simple. You see that it returns an array of 440 +lines of text. We know from our analysis that the table header is on the fourth line (third line for +Linux), so the data starts on the next line. + +```powershell +PS> $lines = netstat -a +PS> $lines.count +440 +PS> $lines[3] + Proto Local Address Foreign Address State +PS> $lines[4] + TCP 0.0.0.0:80 0.0.0.0:0 LISTENING +``` + +To parse the rows into the individual columns of data we need to use the space character as a +delimiter to split the line. Since the number of spaces between columns is variable, the split +operation creates empty fields between the data. We can filter those empty fields out with a +`Where-Object`. For example: + +```powershell +$columns = ($lines[4] -split ' ').Trim() | Where-Object {$_ } +$columns +TCP +0.0.0.0:80 +0.0.0.0:0 +LISTENING +``` + +In this example, the `Trim()` method trims off leading and trailing spaces. This ensures that the +fields between the columns become empty strings. + +## Step 4 - Output the object + +The only thing left to do now is to create a PowerShell object that contains the parsed data. Let's +put this all together. + +```powershell +function parseNetstat { + param([object[]]$Lines) + + if ($IsWindows) { + $skip = 4 + } else { + $skip = 3 + } + + $Lines | Select-Object -Skip $skip | ForEach-Object { + $columns = ($_ -split ' ').Trim() | Where-Object {$_ } + [pscustomobject]@{ + Protocol = $columns[0] + LocalAddress = $columns[1] + RemoteAddress = $columns[2] + State = $columns[3] + } + } +} + +parseNetstat (netstat) | Select-Object -Last 10 +``` + +For this example, I limit the output to the last 10 rows. + +```powershell-console +Protocol LocalAddress RemoteAddress State +-------- ------------ ------------- ----- +TCP [2600:6c56:7e00:78d:e1e8:756c:d2be:42da]:61001 [2603:1036:303:3000::2]:https TIME_WAIT +TCP [2600:6c56:7e00:78d:e1e8:756c:d2be:42da]:61018 [2603:1030:408::401]:https ESTABLISHED +TCP [2600:6c56:7e00:78d:e1e8:756c:d2be:42da]:61293 [2603:1036:303:3000::2]:https ESTABLISHED +TCP [2600:6c56:7e00:78d:e1e8:756c:d2be:42da]:62640 [2603:1036:303:3c33::2]:https ESTABLISHED +TCP [2600:6c56:7e00:78d:e1e8:756c:d2be:42da]:62643 [2603:1036:303:3c04::2]:https ESTABLISHED +TCP [2600:6c56:7e00:78d:e1e8:756c:d2be:42da]:62659 [2603:1036:303:3050::2]:https TIME_WAIT +TCP [2600:6c56:7e00:78d:e1e8:756c:d2be:42da]:64886 ord37s36-in-x0d:https ESTABLISHED +TCP [2600:6c56:7e00:78d:e1e8:756c:d2be:42da]:64887 [2603:1036:404:8e::2]:https TIME_WAIT +TCP [2600:6c56:7e00:78d:e1e8:756c:d2be:42da]:64901 [2620:1ec:21::18]:https ESTABLISHED +TCP [2600:6c56:7e00:78d:e1e8:756c:d2be:42da]:65492 ord30s21-in-x0e:https TIME_WAIT +``` + +Success! I now have converted text output to a PowerShell object. At this point, this is enough to +become an **Output Handler** for a Crescendo module. + +If we want to get fancier, we can parse the address columns into the IP Address and the Port. That +data is in `$column[1]` and `$column[2]`. To separate the Port from the IP Address we have to +determine if the address is IPv4 or IPv6. The following code handles this: + +```powershell +if ($columns[1].IndexOf('[') -lt 0) { + $laddr = $columns[1].Split(':')[0] + $lport = $columns[1].Split(':')[1] +} else { + $laddr = $columns[1].Split(']:')[0].Trim('[') + $lport = $columns[1].Split(']:')[1] +} +if ($columns[2].IndexOf('[') -lt 0) { + $raddr = $columns[2].Split(':')[0] + $rport = $columns[2].Split(':')[1] +} else { + $raddr = $columns[2].Split(']:')[0].Trim('[') + $rport = $columns[2].Split(']:')[1] +} +``` + +First I check that a column contains an open bracket character (`[`]). If it doesn't, I can split +the string at the colon character (`:`). If not then I need to split is at the string `']:'` and +also trim off the opening bracket. + +After adding that code to the function I can now filter on the port information. For example: + +```powershell + parseNetstat (netstat) | + Where-Object {$_.RemotePort -eq 'https' -and $_.State -eq 'ESTABLISHED'} | + Select-Object LocalAddress, LocalPort, RemoteAddress, RemotePort -Last 10 + +LocalAddress LocalPort RemoteAddress RemotePort +------------ --------- ------------- ---------- +2600:6c56:7e00:78d:e1e8:756c:d2be:42da 55643 2603:1036:303:3c1d::2 https +2600:6c56:7e00:78d:e1e8:756c:d2be:42da 59703 2620:1ec:21::18 https +2600:6c56:7e00:78d:e1e8:756c:d2be:42da 59708 2603:1036:303:3c1d::2 https +2600:6c56:7e00:78d:e1e8:756c:d2be:42da 60834 2620:1ec:42::132 https +2600:6c56:7e00:78d:e1e8:756c:d2be:42da 60835 2603:1036:303:3c1c::2 https +2600:6c56:7e00:78d:e1e8:756c:d2be:42da 61018 2603:1030:408::401 https +2600:6c56:7e00:78d:e1e8:756c:d2be:42da 61293 2603:1036:303:3000::2 https +2600:6c56:7e00:78d:e1e8:756c:d2be:42da 61399 ord30s21-in-x03 https +2600:6c56:7e00:78d:e1e8:756c:d2be:42da 65025 2603:1036:303:3c0a::2 https +2600:6c56:7e00:78d:e1e8:756c:d2be:42da 65053 2603:1036:303:3c0c::2 https +``` + +## Conclusion + +Writing the output parser is the hardest part of wrapping a native command, whether you are using +Crescendo or not. In this post I have used a few simple techniques for extracting data from the +strings. In my next blog post I will take a closer look at a more complex parsing example that I +wrote for my VssAdmin module. + +If you are interested in the final version of the script in this post, you can find it in this +GitHub [Gist][2]. + +<!-- link reference --> +[1]: https://devblogs.microsoft.com/powershell-community/my-crescendo-journey/ +[2]: https://gist.github.com/sdwheeler/0ab90a646d401c2e0de36fac59b7cf65 diff --git a/Posts/2021/10/tfl-secrets.md b/Posts/2021/10/tfl-secrets.md new file mode 100644 index 00000000..2a473b6b --- /dev/null +++ b/Posts/2021/10/tfl-secrets.md @@ -0,0 +1,240 @@ +--- +post_title: How to Use The Secrets Module +username: tfl@psp.co.uk +Categories: PowerShell +tags: SecretManagement, passwords, credentials +Summary: Using The Secrets Modules +--- + +**Q:** I have a bunch of scripts we use in production that make use of Windows credentials. +In some cases, these scripts have an actual password in plain text, while others read the password from an XML file. +Is there a better way? + +**A:** Scripts with high-privilege account passwords in plain text is not a good idea. +There are several methods you can use to improve the security of credentials handling. +One great way is to use the SecretManagement and SecretStore modules from the PowerShell Gallery. + +## What are Secrets? + +Secrets are, in general, passwords you need to access some resource. +It might be the password for a domain administrator that you use to run a command on a remote host. +You want to keep secrets secret, yet you want a great way to use them as needed. + +In my PowerShell books, I use a domain (Reskit.Org) for all my examples. +The password for this mythical domain's Enterprise and Domain administrator is "Pa$$W0rd". +I am not too worried about exposing this password as it is only the password to a few dozen VMs. +This means many of the scripts from my books contain the password in clear text. +While great for books, this is not a best practice in production. + +Over the years there have been numerous attempts at handling secrets. +You could store the secrets in an XML file and import the file when you needed those secrets. +Or, you could force the user to just retype the password every time they want to use it. +Speaking personally - I get tired real fast of typing a long, complex, password time and time again! + +## What are the Secrets Module? + +The developers of this module recognized the challenge that users wanted consistency +in managing secrets with flexibility over which secret store to use. +The solution involves separating secrets management from secrets storage. +So there there are _two_ modules involved: + +* SecretManagement - you use this module in your scripts to make use of secrets. +* SecretStore - this module contains the commands to manage a specific secret storage. + +You also need a vault-specific module which the SecretsStore module accesses. +This layered approach allows you to use any secret store you wish, manage the secrets independently of the physical storage mechanism. +You could, in theory, change the secret store and not need to change your scripts that use the secrets. + +## Installing the Modules + +If you want to follow along with the code and do not fancy cut/paste, I have created a GitHub Gist for the code you see in this article. +You can find it [here](https://gist.github.com/doctordns/b1a06f7002675ec2bf8f710d3c066182). + +```powershell-console +PS> # 1. Discover the modules +PS> Find-Module -Name 'Microsoft.PowerShell.Secret*' | + Format-Table -Wrap -AutoSize + +Version Name Repository Description +------- ---- ---------- ----------- +1.1.0 Microsoft.PowerShell.SecretManagement PSGallery This module provides a convenient way for a user + to store and retrieve secrets. The secrets are + stored in registered extension vaults. An + extension vault can store secrets locally or remotely. + SecretManagement coordinates access to the secrets + through the registered vaults. + Go to GitHub for more information about the module + and to submit issues:https://github.com/powershell/SecretManagement + +1.0.4 Microsoft.PowerShell.SecretStore PSGallery This PowerShell module is an extension vault for the + PowerShell SecretManagement module. + As an extension vault, this module stores secrets to the local + machine based on the current user account context. + The secrets are encrypted on file using .NETCrypto APIs. + A password is required in the default configuration. + The configuration can be changed with the provided cmdlets. + Go to GitHub for more information about this module + and to submit issues: https:////github.com//powershell//SecretStore + +PS> # 2. Install both modules +PS> Install-Module -Name $Names -Force -AllowClobber +``` +When you install the module using `Install-Module` you see no output (unless you use the `-Verbose` switch). +You can always use `Get-Module` to check that you have installed these new (to you) modules. + +## Discovering the commands available to you +Once you have thess two modules installed, you can discover the commands in each module: + +```powershell-console + +PS> # 3. Examine them +PS>PS> Get-Module -Name Microsoft*.Secret* -ListAvailable | + Format-Table -Property ModuleType, Version, Name, ExportedCmdlets + +ModuleType Version Name ExportedCmdlets +---------- ------- ---- --------------- + Binary 1.1.0 Microsoft.PowerShell.SecretManagement {[Register-SecretVault, Register-SecretVault], + [Unregister-SecretVault, Unregister-SecretVault], [Get-SecretVault, + Get-SecretVault], [Set-SecretVaultDefault, Set-SecretVaultDefault], + [Test-SecretVault, Test-SecretVault], [Set-Secret, Set-Secret], + [Set-SecretInfo, Set-SecretInfo], [Get-Secret, Get-Secret], + [Get-SecretInfo, Get-SecretInfo], [Remove-Secret, Remove-Secret], + [Unlock-SecretVault, Unlock-SecretVault]} + Binary 1.0.5 Microsoft.PowerShell.SecretStore {[Unlock-SecretStore, Unlock-SecretStore], [Set-SecretStorePassword, + Set-SecretStorePassword], [Get-SecretStoreConfiguration, + Get-SecretStoreConfiguration], [Set-SecretStoreConfiguration, + Set-SecretStoreConfiguration], [Reset-SecretStore, + Reset-SecretStore]} + +``` + +As you can see, both modules have a number of commands you may need to use to manage secrets for your environment. +Also - depending on your screen width you may find your output is slightly diffetrent although it should contain the same information. + +## Registering and viewing a secret vault + +After you have the two modules installed, your next step is to register a secret vault. +There are several vault options you can take advantage of, for this post, I'll use the built-in default vault. +You configure the default vault like this: + +```powershell-console +PS> # 4. Register the default secrets provider +PS> $Mod = 'Microsoft.PowerShell.SecretStore' +PS> Register-SecretVault -Name RKSecrets -ModuleName $Mod -DefaultVault +PS> Get-SecretVault + +Name ModuleName IsDefaultVault +---- ---------- -------------- +RKSecrets Microsoft.PowerShell.SecretStore True +``` +Like the previous step, registering the vault does not create any output by default. +You can view the vault you just created by using the `Get-SecretVault` command. + +## Setting a secret + +To create a new secret in your secret vault, you use the `Set-Secret` command, like this: + +```powershell-console + +PS> # 4. Register the default secrets provider +PS> Import-Module -Name 'Microsoft.PowerShell.SecretManagement' +PS> Import-Module -Name 'Microsoft.PowerShell.SecretStore' +PS> $Mod = 'Microsoft.PowerShell.SecretStore' +PS> Register-SecretVault -Name RKSecrets -ModuleName $Mod -DefaultVault +PS> # 5. View Secret vault +PS> Get-SecretVault + +Name ModuleName IsDefaultVault +---- ---------- -------------- +RKSecrets Microsoft.PowerShell.SecretStore True + +PS C:\Foo> # 6. Set the Admin password secret for Reskit forest +PS C:\Foo> Set-Secret -Name ReskitAdmin -Secret 'Pa$$w0rd' +Creating a new RKSecrets vault. A password is required by the current store configuration. +Enter password: +********** +Enter password again for verification: +********** +``` + +This code fragment explicitly loads both of the downloaded modules. +If you use PowerShell module automatic loading, this is unnecessary. + +Also, the first time you use `Set-Secret` to create a secret, the cmdlet prompts for a vault password. +Note this password isd NOT stored in the AD - so don't forget it!!! + +As an aside - I hope you noticed the bad practice in the above code - using a clear text password in a script file. +A better approach to this _for production coding_ would be to use `Read-Host` to have the password passed in. +In this case, you see the actual password I set, and later see that this password was indeed saved and retreived correctly. + +## Using secrets stored in your secret vault + +Now that you have set a password in the RKSecrets vault, you can use the `Get-Secret` cmdlet to retrieve the secret. +As you can see here, although you set a plain text password, `Get-Secret` returns the secret as a secure string. + +```powershell-console +PS> # 7. Create a credential object using the secet +PS> $User = 'Reskit\\Administrator' +PS> $PwSS = Get-Secret ReskitAdmin +PS> $Cred = [System.Management.Automation.PSCredential]::New($User,$PwSS) +PS> # 8. Let's cheat and see what the password is first. +PS> $PW = $Cred.GetNetworkCredential().Password +PS> "Password for this credential is [$PW]" +Password for this credential is [Pa$$w0rd] +PS> # 9. Using the credential against DC1 +PS> $Cmd = {hostname.exe} +PS> Invoke-Command -ComputerName DC1 -Credential $Cred -ScriptBlock $Cmd +DC1 +``` + +As you can see, it is straightforward to create a new credential object using a password retrieved from the vault. +This code creates a new PSCredential object, because that is what PowerShell cmdlets use to authenticate remoting sessions. +You can use the credential object's `GetNetworkCredential()` method to retrieve the plain text password. + +If you are running this code, the first time you create a vault, the secrets module requires you to specify a vault password. +Depending on what sequence of commands you enter and how quickly, you may be asked to re-enter your vault password. + +## Using Metadata + +If you have a large numbers of secrets to manage, you can add additional metadata to help you keep track of the secrets you set. +Metadata is a simple hash table containing the metadata you wish to apply to a secret. +Each item in the hash table is a key-value pair. +The keys can be anything you wish such as the purpose of the script and the script author. +You use `Set-Secret` to add metadata to an existing (or new) secret. +To set the metadata, you can use the `Get-SecretInfo` cmdlet. +Creating and using metadata looks like this: + +```powershell-console +PS> # 10. Setting metadata +PS> Set-Secret -Name ReskitAdmin -Secret 'Pa$$w0rd' -Metadata @{Purpose="Reskit.Org Enterprise\\Domain Admin PW"} +PS> Get-SecretInfo -Name ReskitAdmin | Select-Object -Property Name, Metadata + +Name Metadata +---- -------- +ReskitAdmin {[Purpose, Reskit.Org Enterprise/Domain Admin PW]} + +PS> # 11. Updating the metadata +PS> Set-SecretInfo -Name ReskitAdmin -Metadata @{Author = 'DoctorDNS@Gmail.Com'; + Purpose="Reskit.Org Enterprise\\Domain Admin PW"} +PS> # 12. View secret information with metadata +PS> Get-SecretInfo -Name ReskitAdmin | Select-Object -Property Name, Metadata + +Name Metadata +---- -------- +ReskitAdmin {[Purpose, Reskit.Org Enterprise\\Domain Admin PW], + [Author, DoctorDNS@Gmail.Com]} +``` + +As noted, Metadata can be any key-value pair you wish to add to the secret. +In this case, the code set two metadata items: the purpose of the secret and its author. +Feel free to add whatever metadata makes sense to you and your organization. + + +## Summary + +The two secrets modules provide a great way to use secrets in your PowerShell scripts and keep the secrets secure. +These two modules work both with Windows PowerShell and PowerShell 7. +The default secrets vault works well enough for most cases, but you have options. +If there is an interest, I can create a further blog post to look at using different secret vaults. + +So stop using plain text secrets in your PowerShell scripts and use the secrets modules. diff --git a/Posts/2021/11/tfl-formatenumeration.md b/Posts/2021/11/tfl-formatenumeration.md new file mode 100644 index 00000000..6dac5306 --- /dev/null +++ b/Posts/2021/11/tfl-formatenumeration.md @@ -0,0 +1,153 @@ +--- +post_title: How to Use $FormatEnumerationLimit +username: tfl@psp.co.uk +Categories: PowerShell +tags: PowerShell, Format, FormatEnumerationLimit variable +Summary: Using The $FormatEnumerationLimit variable in PowerShell +--- + +**Q:** When I format an object where a property contains more than 4 objects, I never see the extra property values. How can I fix that? + +**A:** Use the `$FormatEnumerationLimit` variable. + +This query is one I hear in many PowerShell support forums, and I have encountered this issue a lot over the years. +What happens is that you issue a command to return objects, for example `Get-Process`. +The `Get-*` cmdlets return objects which can contain properties that are arrays of values, not just a single value. +When you pipe those objects to `Format-Table`, by default, PowerShell only shows you the first four. + +Let me illustrate what this looks like (by default): + +```powershell-console +PS> Get-Process -Name pwsh | Format-Table -Property ProcessName, Modules + +ProcessName Modules +----------- ------- +pwsh {System.Diagnostics.ProcessModule (pwsh.exe), + System.Diagnostics.ProcessModule (ntdll.dll), + System.Diagnostics.ProcessModule (KERNEL32.DLL), + System.Diagnostics.ProcessModule (KERNELBASE.dll)…} +``` + +This output shows PowerShell getting the process object for `Pwsh.exe` and then passing it to `Format-Table`, which outputs the process name and the modules used by that process. +However, as you can see, PowerShell shows only four modules shown followed by "…" (also known as an ellipsis). +The ellipsis tells you that there are more values in this property, except PowerShell does not show them. + +If you know the `Format-Table` command, you might be tempted to use the `-Wrap` or the `-AutoSize` parameters, but these would not help. +It turns out there is no parameter for `Format-Table` or `Format-List` to control this. +The trick is to use the `$FormatEnumerationLimit` variable and assign it a higher value. + +The `$FormatEnumerationLimit` automatic variable tells PowerShell and the formatting cmdlets how many occurrences to include in the formatted output. +By default, PowerShell sets this variable to four at startup. +And that is why you see just four processes in output (by default). + +With PowerShell, you can adjust this limit in a script or a profile file. +When you change the value, PowerShell outputs more occurrences, up to the limit you set in `$FormatEnumerationLimit`. +Like this: + +```powershell-console +PS > $FormatEnumerationLimit = 8 +PS > Get-Process -Name PWSH | Format-Table -Property ProcessName, Modules + +ProcessName Modules +----------- ------- +pwsh {System.Diagnostics.ProcessModule (pwsh.exe), + System.Diagnostics.ProcessModule (ntdll.dll), + System.Diagnostics.ProcessModule (KERNEL32.DLL), + System.Diagnostics.ProcessModule (KERNELBASE.dll), + System.Diagnostics.ProcessModule (apphelp.dll), + System.Diagnostics.ProcessModule (USER32.dll), + System.Diagnostics.ProcessModule (win32u.dll), + System.Diagnostics.ProcessModule (GDI32.dll)…} +``` + +In the above output, you can see output for eight modules. +In writing this, there are actually 239 actual modules for the PowerShell process. +If you need to see all the modules, you could set `$FormatEnumerationLimit` to a larger number (e.g. 999) in the shell. +Alternatively, if you set `$FormatEnumerationLimit` to -1, PowerShell displays all occurrences, which may be more than you want in most cases! +I set the limit to 99 in my profile file and that is usually more than sufficient. + +## Scoping of $FormatEnumerationLimit + +One interesting thing I found is that `$FormatEnumerationLimit` is scoped differently to my expectations. +If you use a format command within a function or script (a child of the global scope), the command only uses the value from the global scope. + +The following code contains a function to illustrate the issue: + +```powershell +function Test-FormatLimitLocal +{ + # Change format enum limit + "In Function, limit is: [$FormatEnumerationLimit]" + $FormatEnumerationLimit = 1 + "After changing: [$FormatEnumerationLimit]" + Get-Process | Select-Object -Property Name, Threads -First 4 +} +``` + +You might think that this code would display the first thread in each of the first four processes. +You might, but you would be wrong, as you can see here: + +```powershell-console +PS> # Here show the value and call the functin +PS> "Before calling: [$FormatEnumerationLimit]" +Before calling: [4] +PS> Test-FormatLimitLocal +In Function, limit is: [4] +After changing: [1] + +Name Threads +---- ------- +AggregatorHost {5240} +ApplicationFrameHost {16968, 2848} +AppVShNotify {9164} +Atom.SDK.WindowsService {4064, 4908, 4912, 19144…} +``` + +As you can see from this output, the final process shows FOUR threads not ONE. +This is because PowerShell seems to only use the globally scoped value, not the locally scoped copy. +To get around this curious scoping, you can re-write the function like this: + +```powershell + +function Test-FormatLimitGlobal +{ + # Change format enum limit Globally + $Old = $Global:FormatEnumerationLimit + $Global:FormatEnumerationLimit = 1 + "After changing: [$Global:FormatEnumerationLimit]" + Get-Process | Select-Object -Property Name, Threads -First 4 + # Change it back + $Global:FormatEnumerationLimit = $Old +} +``` + +When you call the updated function, it now operates more as you might wish, like this: + +```powershell-console +PS> # View the value +PS> "Before calling: [$FormatEnumerationLimit]" +Before calling: [4]# +PS> # Now call the updated function +PS> Test-FormatLimitGlobal +After changing: [1] + +Name Threads +---- ------- +AggregatorHost {5240} +ApplicationFrameHost {16968…} +AppVShNotify {9164} +Atom.SDK.WindowsService {4064…} +``` + +So, with some careful updating of the global variable, you can get the desired result. +In general, I teach my students to avoid manipulating global variables from within a script or a function (unless you know what you are doing). +If you need to make changes to any global variable to make a function or script do what you want, ensure you know how to revert the variable to its original value. + +I am unclear whether this is a bug or a feature! To that end, I submitted a [feature request](https://github.com/PowerShell/PowerShell/issues/16360) in the PowerShell source repository. Feel free to add your opinion in the comments or upvote it if you want to see it added. + +## Summary + +The `$FormatEnumerationLimit` variable is a neat feature of PowerShell that allows you to see more occurrences when using `Format-Table`. +But remember: if you are using this variable in a function or a script, you should be aware of the scoping issue. + +You can read more about `$FormatEnumerationLimit`, and other preference variables in [about_Preference_Variables](https://docs.microsoft.com/powershell/module/microsoft.powershell.core/about/about_preference_variables#formatenumerationlimit). diff --git a/Posts/2021/12/jon-1.png b/Posts/2021/12/jon-1.png new file mode 100644 index 00000000..86cc1990 Binary files /dev/null and b/Posts/2021/12/jon-1.png differ diff --git a/Posts/2021/12/jon-2.png b/Posts/2021/12/jon-2.png new file mode 100644 index 00000000..c33a29d3 Binary files /dev/null and b/Posts/2021/12/jon-2.png differ diff --git a/Posts/2021/12/jon-preferences.md b/Posts/2021/12/jon-preferences.md new file mode 100644 index 00000000..1cb83491 --- /dev/null +++ b/Posts/2021/12/jon-preferences.md @@ -0,0 +1,258 @@ +--- +post_title: On Preferences and Scopes +username: j@mobulaconsulting.com +Categories: PowerShell +tags: PowerShell, Preference Variables, Verbose, Progress, ErrorAction +Summary: How to have more control of preferences in functions and the role of modules on inheritance. +--- + +# Progress in PowerShell: a tale of Verbosity and other preferences with lessons in Scopes and Proxies thrown in + +It started, as these things often do, with someone complaining. In PowerShell Version 7.2 the output of `Invoke-WebRequest -Verbose` and `Invoke-RestMethod -Verbose` look like this: + +```powershell-console +VERBOSE: GET with 0-byte payload +``` + +In all the earlier versions they look like the version below , which is more helpful when you're trying to debug code that builds URIs: + +```powershell-console +VERBOSE: GET https://google.com/ with 0-byte payload +``` + +A *proxy function* will fix that. If two commands have the same name an alias beats a function, +which beats a cmdlet, which beats an external program. You can specify the full path +to an external program or cmdlet - for example `Microsoft.PowerShell.Utility\Invoke-RestMethod` +so an `Invoke-RestMethod` *function* can act as a *proxy* for the cmdlet, anything which calls `Invoke-RestMethod` will go to the function, which calls the cmdlet with its fully qualified name. PowerShell even has a mechanism to create the function's code: + +```powershell +$cmd = Get-Command Invoke-RestMethod +$MetaData = New-Object System.Management.Automation.CommandMetaData $cmd +[System.Management.Automation.ProxyCommand]::create($MetaData) | clip +``` + +(I don't carry those 3 lines in my head, when I need them I refer to a script Jeffrey Snover wrote long ago, a newer version is [on the PowerShell Gallery](https://www.powershellgallery.com/packages/MetaProgramming/1.0.0.2) .) +I added extra `Write-Verbose` calls and tidied up the autogenerated code and posted the result [as a gist](https://gist.github.com/jhoneill/f8ddd4e4e0a25c22d77749166d6f14fe). +A module I'm working has lots of calls to `Invoke-RestMethod` +but the proxy function wouldn't see that I'd specified `-Verbose`. So I needed to investigate. + +The `-Verbose` switch sets the value of `$VerbosePreference` in the function being called; +if you thought setting the global `$VerbosePreference` to `continue` was the same as a running +everything with `-Verbose`, trying the following might surprise you: + +```powershell +$VerbosePreference="Continue" +Invoke-WebRequest "https://google.com" -OutFile delete.me +Copy-Item delete.me delete.too +``` + +`Invoke-WebRequest` heeds the preference, but `Copy-Item` only prints a message if the `-Verbose` *switch* is specified. + +My proxy function would print a verbose message if run with `-Verbose`, it *would* heed the global preference-variable but not the switch passed to the function that called it. +The `-Confirm`, `-Debug`, `-ErrorAction`, `-InformationAction`, `-Verbose`, `-WarningAction`, and `-WhatIf` switches *all* +set the corresponding preference-variable inside a function but that wasn't being inherited when the functions in my module called the proxy function. +I could reproduce this with the two functions below. First, I loaded them from a +single .PSM1 file (and things were the same if I pasted them in at a PowerShell prompt) + +```powershell +function one { + [cmdletBinding()] + param() + Write-Verbose "One calls two" + two + Write-Verbose "Two returned" +} + +function two { + [cmdletBinding()] + param() + Write-Verbose "Two has a message" +} +``` + +If I load this, I get three messages: + +```powershell-console +VERBOSE: One calls two +VERBOSE: Two has a message +VERBOSE: Two returned +``` + +Things change if `One` is in it’s own module + +![Screen shot showing the difference if the calling function is an module](./jon-1.png) + +`One -verbose` now just produces two messages: + +```powershell-console +VERBOSE: One calls two +VERBOSE: two returned +``` + +Setting the global preference-variable at the prompt returns all 3 - because function `two` sees it. +It is common to assume that *a function inherits the variables from whatever called it*; we can see that working with simpler functions + +```powershell +function three { + $e="ewe" + four +} + +function four { + $e +} +``` + +If I set “e” and run `three` from the prompt + +```powershell-console +> $e = "eye" +> three +ewe +``` + +the assumption holds; and a lot of documentation stops there, but if I import `three` from a module + +```powershell-console +> three +eye +``` + +The inheritance assumption is qualified by a rule that says *what happens in a module stays in the module*. +The question is *what can we do about it ?* + +Before it dawned this was a *scopes* thing and not just `-Verbose`, a search brought me [a clue](https://deangrant.me/2015/10/03/inheriting-verbose-preference-in-windows-powershell-module-functions/); the solution is to change function `two` (or my proxy function) as follows: + +```powershell +function two { + [cmdletBinding()] + param($VerbosePreference = $PSCmdlet.GetVariableValue('VerbosePreference') + Write-Verbose "Two has a message" +} +``` + +We can only use `$PSCmdlet` if we have either `[cmdletBinding()]` or a `[parameter()]` decoration - just a side note on that, if you paste in + +```powershell +function five { param ($p) } +function six { param ([parameter()]$p) } +``` + +when you try to tab-complete parameters for `five` and `six`, you’ll see `six` gets all +the common parameters but `five` does not - `[parameter()]` is an implicit `[cmdletBinding()]`, although it's still good to write the latter explicitly. + +`$PSCmdlet` is available *when the function is setting up its variables*. Inside the function we'd never replace `$x` with the long-winded +`$PSCmdlet.GetVariableValue("x")`, but *in a parameter* it is “use the value from the scope that called you - even if that scope *is* a module”. + +*Now* the called function inherits the preference from its caller. If we specify `-Verbose` it takes precedence, so nothing breaks copying in `$VerbosePreference`. The same thing applies to `-Confirm` and `-WhatIf`. + +```powershell +function seven { + [cmdletBinding(SupportsShouldProcess=$true)] + param() + eight + Write-Host "Something Safe " +} + +function eight { + [cmdletBinding(SupportsShouldProcess=$true)] + param() + if ($PSCmdlet.ShouldProcess("This","Do you want to do")) { + Write-Host "Something dangerous" + } +} +``` + +If `seven` and `eight` are loaded from the same psm1 file + +```powershell-console +seven -Confirm + +Confirm +Are you sure you want to perform this action? +Performing the operation "Do you want to do" on target "This". +[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is"Y"): n + +Something Safe +``` + +But, if `seven` loads from its own module … + +```powershell-console +seven -Confirm +Something dangerous +Something Safe +``` + +Modifying the parameters in `eight` with the code below restores the confirmation + +```powershell +param($ConfirmPreference = $PSCmdlet.GetVariableValue('ConfirmPreference')) +``` + +`-confirm` sets `$confirmPreference` to `low` which triggers `$PSCmdlet.ShouldProcess` to prompt the user. I treat `-Force` as an extra preference and my functions typically have +`if ($Force -or $PSCmdlet.ShouldProcess(...` +to ensure the a command can be run non-interactively, even if its impact is set higher than `$ConfirmPreference`. + +The example above might lead you to think `-Confirm` and `-WhatIf` *should* be inherited, but this would cause a problem with scopes at the `script` level (which apply module-wide). Suppose the module that contained seven read like this: + +```powershell +$ConfirmPreference = 'None' + +function seven { +... +} +``` + +Any cmdlet that would normally ask for confirmation inside `seven` will run silently. But what if the module containing `eight` sets that option to something different? If a function overrides something from its own script-scope, then only things which share that scope see the change, which seems logical. Thinking of functions' scopes as *children* of their module's scope which in turn is usually a child of the global scope means we can say things pass down their branch of the scope “tree” but don't jump *between* branches. + +Before leaving `$ConfirmPreference`, I have been think about changing it inside a function, so it drops to `low` if +many items are being updated. I’m not sure that's a great idea because coming to *rely* on a +prompt and which isn't there *reliably*, will lead to trouble. + +The title said this is about *Progress*. PowerShell 7.2 has an optional new way of +displaying the progress bar, but the things it relies on are a bit flaky in the +Integrated-Shell in Visual Studio Code +(things improve if you use `Write-Progress -completed`) and `Invoke-WebRequest` downloading 100 log files becomes a mess. +There is no `-Progress` common parameter, but the day before I started on the `-Verbose` problem, I had found that adding +`$ProgressPreference` as a parameter worked, so it makes sense to do it the same way. I inserted the following parameter into the function that calls `Invoke-WebRequest`. Unlike the examples above, which might be tagged with `[Parameter(DontShow)]` I want this to tab-complete the possible values, so it is marked with the `ActionPreference` type: + +```powershell +[ActionPreference]$ProgressPreference = $PSCmdlet.GetVariableValue('ProgressPreference') +``` + +And I will also want *that* to inherit into my proxy function. +With that in place, I can use the code below to call my function and replace the byte-by-byte progress indicator that `Invoke-WebRequest` displays with my own file-by-file one: + +```powershell +$count = 0 +ForEach ($f in $file) { + Write-Progress "Downloading" -PercentComplete ($count/$file.Count) + Myfunction $f -ProgressPreference SilentlyContinue + $count += 100 +} +Write-Progress "Downloading" -Completed +``` + +Quick tip: adding 100 to the counter each time (instead of adding 1) removes the need to multiply by 100 for a percentage. + +Since I mentioned `ErrorAction` above, before finishing I wanted to share one last tip about preferences: + +```powershell +function nine { + [cmdletBinding()] + param ($Name) + if (-not $Name) {throw "Name is required"} + Write-Host "Deleting $Name*" +} +``` + +The “dangerous” line in the example above never runs if `$n` is empty, right? Wrong, actually. + +![Screen shot showing the effect of -ErrorAction on throw](./jon-2.png) + +Specifying `-ErrorAction` prevents the `throw` statement throwing so + +`nine "" -ErrorAction SilentlyContinue` would mean the dangerous code *is* run. + +When it is acting as a “fence” around dangerous code, it is worth putting a `return` after `throw`. diff --git a/Posts/2021/12/media/tfl-preview/after.png b/Posts/2021/12/media/tfl-preview/after.png new file mode 100644 index 00000000..9ba76088 Binary files /dev/null and b/Posts/2021/12/media/tfl-preview/after.png differ diff --git a/Posts/2021/12/media/tfl-preview/before.png b/Posts/2021/12/media/tfl-preview/before.png new file mode 100644 index 00000000..a670465f Binary files /dev/null and b/Posts/2021/12/media/tfl-preview/before.png differ diff --git a/Posts/2021/12/tfl-params.md b/Posts/2021/12/tfl-params.md new file mode 100644 index 00000000..abc77bd6 --- /dev/null +++ b/Posts/2021/12/tfl-params.md @@ -0,0 +1,94 @@ +--- +post_title: How to Use $PSDefaultParameterValues +username: tfl@psp.co.uk +Categories: PowerShell +tags: PowerShell, Default Parameter values, parameters +Summary: Using The $PSDefaultParameterValues automatic variable +--- + +**Q:** When I use cmdlets like `Receive-Job` and `Format-Table`, how do I change default values of the **Keep** and **Wrap** parameters? + +**A:** Use the `$PSDefaultValues` automatic variable. + +When I first discovered PowerShell's background jobs feature, I would use `Receive-Job` to view job output - only to discover it's no longer there. +And almost too often to count, I pipe objects to `Format-Table` cmdlet only to get truncated output because I forgot to use `-Wrap`. +I'm sure you all have parameters whose default value you would gladly change - at least for your environment. + +I'm sure you have seen this (and know how to use **Wrap**), like this: + +```powershell-console +PS> # Default output in a narrow terminal window. +PS> Get-Service | Format-Table -Property Name, Status, Description + +Name Status Description +---- ------ ----------- +AarSvc_f88db Running Runtime for activating conversational … +AJRouter Stopped Routes AllJoyn messages for the local … +ALG Stopped Provides support for 3rd party protoco… +AppHostSvc Running Provides administrative services for I… +... +PS > # Versus this using -Wrap +PS > Get-Service | Format-Table -Property Name, Status, Description -Wrap + +Name Status Description +---- ------ ----------- +AarSvc_f88db Running Runtime for activating conversational agent + applications +AJRouter Stopped Routes AllJoyn messages for the local + AllJoyn clients. If this service is stopped + the AllJoyn clients that do not have their + own bundled routers will be unable to run. +ALG Stopped Provides support for 3rd party protocol + plug-ins for Internet Connection Sharing +AppHostSvc Running Provides administrative services for IIS, + for example configuration history and + Application Pool account mapping. If this + service is stopped, configuration history + and locking down files or directories with + Application Pool specific Access Control + Entries will not work. + +``` + +So, the question is: how to tell PowerShell to always use `-Wrap` when using `Format-Table` or `Format-List`? +It turns out there is a very simple way: use the `$PSDefaultParameters` automatic variable. + +## The `$PSDefaultParameters` automatic variable + +When PowerShell (and Windows PowerShell) starts, it creates the `$PSDefaultParameters` automatic variable. +The variable has a type: **System.Management.Automation.DefaultParameterDictionary**. +In other words, the variable is a Powershell hash table. +By default, the variable is empty when you start PowerShell. + +Each entry in this hash table defines a cmdlet, a parameter and a default value for that parameter. +The hash table key is the name of the cmdlet, followed by a colon (`:`), and then the name of the parameter. +The hash table value for this key is the new default value for the parameter. + +If you wanted, for example, to always use **-Wrap** for the `Format-*` cmdlets, you could do this: + +```PowerShell +$PSDefaultParameterValues.Add('Format-*:Wrap', $True) +``` +## Persist the change in your profile +Any change you make to the `$PSDefaultParameterValues` variable is only applicable for the current session. +And the variable is subject to normal scoping rules - so changing the value in a script does not affect the session as a whole. +That means that if you want these changes to occur every time you start a PowerShell console, then you add the appropriate statements in your profile. + +On my development box, I use the following snippet inside my `$PROFILE` script: + +```powerShell +$PSDefaultParameterValues.Add('Format-*:AutoSize', $true) +$PSDefaultParameterValues.Add('Format-*:Wrap', $true) +$PSDefaultParameterValues.Add('Receive-Job:Keep', $true) +``` + +## Summary + +The `$PSDefaultParameterValues` automatic variable is a great tool to help you specify specific values for cmdlet parameters. +You can specify one or more cmdlets by using wild cards in the hash table's key. +Remember that the hash table key is the name of the cmdlet(s), a colon, and then the parameter's name. +Also, the hash table value is the new "default" value for that parameter (and for the specified cmdlet(s)). + +You can read more about `$PSDefaultParameterValues`, and other preference variables in [about_Preference_Variables](https://docs.microsoft.com/powershell/module/microsoft.powershell.core/about/about_preference_variables#psdefaultparametervalues). +And for more details of parameter default values, see the [about_Parameters_Default_Values help file](https://docs.microsoft.com/powershell/module/microsoft.powershell.core/about/about_parameters_default_values). + diff --git a/Posts/2021/12/tfl-preview.md b/Posts/2021/12/tfl-preview.md new file mode 100644 index 00000000..41d2e5a0 --- /dev/null +++ b/Posts/2021/12/tfl-preview.md @@ -0,0 +1,62 @@ +--- +post_title: How to Preview PowerShell Scripts In PowerShell +username: tfl@psp.co.uk +Categories: PowerShell +tags: PowerShell, Explorer, Explorer Preview +Summary: How to Preview PowerShell .PS1, .PSD1, .PSD1 files inside Windows Explorer. +--- + +**Q:** When I use Windows Explorer and select a PowerShell script file - I do not see the script in the preview window. Can I fix that? + +**A:** You can make a few simple registry updates and do just what you want! + +At some time in the deep and distant past, Windows Explorer gained the preview pane feature. +The idea is simple: you select a file in Explorer and Windows shows you a preview of the file in a separate pane. +I love this feature, although when clicking on a Word document, it could take a few moments before I could view. +And if you are viewing a file, it was "open" and you could not delete it in a separate window! +A great feature albeit with some minor side effects - so it makes sense that this is turned off by default. +But you can easily turn it back on! + +## Microsoft PowerToys to the rescue?? + +You can use [Microsoft's Power Toys for Windows](https://docs.microsoft.com/windows/powertoys) to enable Explorer to preview more file types. +I love these tools - and have them installed on my computers. +Sadly, PowerToys currently does not enable previewing of PowerShell files. + +## Enabling Preview in Windows Explorer + +As I mentioned above, file preview within Windows Explorer is disabled by default. +To turn this on, use Explorer's View menu and select preview. +I leave the details of how to set this up as an exercise for the user. +As a small aside, this setting gets reset each time you upgrade Windows - as a Windows Insider, I have to reset this with each new build I take. :-( + +## Enabling Preview of .PS1/.PSD1/.PSM1 files + +Once you enable preview mode in Explorer as shown above, when you select a `.PS1` file - you see something like this: + +![Viewing a .PS1 file in Preview](./media/tfl-preview/before.png) + +There is currently no mechanism in Explorer to change the list of file types to be displayed. +Fortunately, there is a straightforward mechanism that involves setting a registry key value. +To enable Explorer to display the relevant files, you can use the following script fragment: + +```powershell +# Set path variables for PowerShell file types +$Path1 = 'Registry::HKEY_CLASSES_ROOT\\.ps1' +$Path2 = 'Registry::HKEY_CLASSES_ROOT\\.psm1' +$Path3 = 'Registry::HKEY_CLASSES_ROOT\\.psd1' + +# Enable preview of those file types +New-ItemProperty -Path $Path1 -Name PerceivedType -PropertyType String -Value 'text' +New-ItemProperty -Path $Path2 -Name PerceivedType -PropertyType String -Value 'text' +New-ItemProperty -Path $Path3 -Name PerceivedType -PropertyType String -Value 'text' +``` + +## Result! + +Once you run this script, Explorer displays the script file, Explorer now looks like this: + +![Viewing a .PS1 file in Preview after updating the registry](./media/tfl-preview/after.png) + +That's it - a small change to the registry and I can now preview PowerShell files. +Very handy! diff --git a/Posts/2022/07/PasswordExpiryNotificationUsingTeamsandGraphAPI.md b/Posts/2022/07/PasswordExpiryNotificationUsingTeamsandGraphAPI.md new file mode 100644 index 00000000..01e707ac --- /dev/null +++ b/Posts/2022/07/PasswordExpiryNotificationUsingTeamsandGraphAPI.md @@ -0,0 +1,334 @@ +--- +post_title: Password Expiry Notification Using Teams and Graph API +username: farisnt +categories: PowerShell +tags: GraphAPI, Teams, PowerShell +summary: This post's intent is to show how to use Graph API through PowerShell to send a Teams message. +--- + +**Q**: How do I send a password expiration notification to a user using Teams chat? + +**A**: Not only can you send the password notification, but you can use PowerShell with the Teams Graph API to send any message to a Teams user. + +But first, let's talk about Graph API, so we are all on the same page. + +## What is the Graph API? + +Microsoft had a different endpoint for each cloud service. +This makes it hard for the admin as it needs knowledge of each endpoint API URI and manages the authentication and authorization separately. +So Microsoft came up with the Graph API as a one-stop shop to manage all the cloud services using a single endpoint, authentication, and a scoped authorization. + +[Microsoft Graph](https://docs.microsoft.com/graph/overview) is the gateway to read data from a wide range of Microsoft services, including Azure Active Directory, Teams, OneDrive…etc. +You can get the data using a single module and a single interface. + +Microsoft Graph API supports modern authentication protocols such as access token, certificate, and browser authentication. + +> You can read more about the Graph API available endpoint from the [Microsoft Graph REST API Endpoint v1.0 Reference](https://docs.microsoft.com/graph/api/overview). + +## Downloading Graph API PowerShell Module + +You can download Microsoft Graph PowerShell Module by running the following command. + +```powershell +Install-Module -Name Microsoft.Graph +``` + +Microsoft PowerShell Graph Module SDK is cross-platform and supports Windows, macOS, and Linux. + +## Connecting To Graph API Using PowerShell + +Unlike other modules such as the AzureAD, ExchangeOnline, etc. +where the admin needs only to connect with the right credentials and have full access, the graph has a different approach. + +When connecting to the Graph API, you need to specify the scope of permissions or, let's say, declare the required permissions that are used during the script execution. +The script fails if it tries to perform an action that was not in the scope. + +For example, if the script needs to read all user data in the azure directory, it's not enough just to connect to read all the data, even if the user credentials are for the global admin for the tenant. +Instead, you must declare and specify that you will connect and use the `User.Read.All` permission. + +To connect to Graph API with the required scope, use the following: + +```powershell-console +PS> $Scope=@('User.Read.All','User.ReadWrite.All') +PS> Connect-MgGraph -Scopes $Scope + +Welcome To Microsoft Graph! +``` + +To check which identity is used during the connecting with the Graph API along with the used scope, use the `Get-MgContext` cmdlet. + +```powershell-console +PS> Get-MgContext + +ClientId : 2ee82eec-204b-204b-204b-e55eec26bf5a +TenantId : 14d82eec-4d5a-4d5a-4d5a-26bf5ae55eec +CertificateThumbprint : +Scopes : {User.Read.All, User.ReadWrite.All…} +AuthType : Delegated +AuthProviderType : InteractiveAuthenticationProvider +CertificateName : +Account : farismalaeb@contoso.com +AppName : Microsoft Graph PowerShell +ContextScope : CurrentUser +Certificate : +PSHostVersion : 2022.6.3 +ClientTimeout : 00:05:00 +``` + +To add additional permission to the scope, rerun `Connect-MgGraph`, setting the new scope with the **Scope** parameter and connect again. There is no need to specify the same scope already provided. + +So let's assume that an admin connected to the Graph API using the following scope: + +```powershell +$Scope=@('User.Read.All') +Connect-MgGraph -Scopes $Scope +``` + +Later on, the admin wants to add the `User.ReadWrite.All` permission, all the admin needs to do is run the `Connect-MgGraph` and set the new scope + +```powershell +$Scope=@('User.ReadWrite.All') +Connect-MgGraph -Scopes $Scope +``` + +The new scope permission adds to the current one. + +A good starting point in finding out the required permission to execute a certain cmdlet is [Find-MgGraphcommand](https://docs.microsoft.com/powershell/microsoftgraph/find-mg-graph-command) and [Graph Explorer](https://developer.microsoft.com/graph/graph-explorer). + +## Password Expiry Notification Using Teams and Graph API + +### Getting Password Expiration information + +To make this tutorial more fun, let's make a scenario. +Consider a company named Contoso.com. +Contoso.com have an on-premise AD syncing with AAD. The Password Expiration policy is set to 3 months. + +The administrator is looking for a way to send the users a notification through Microsoft Teams chat one week before the password expires, so, how to start? + +To get the password expiration for users, use the following code. +This code reads the **Name**, **EmailAddress**, **UserPrincipalName** and [**msDS-UserPasswordExpiryTimeComputed**](https://docs.microsoft.com/openspecs/windows_protocols/ms-adts/f9e9b7e2-c7ac-4db6-ba38-71d9696981e9). +The **msDS-UserPasswordExpiryTimeComputed** property notes when the user's password expires, check it below. + +```powershell +$DaysToSendWarning = (Get-Date).AddDays(7).ToLongDateString() + +$QueryParameters = @{ + Filter = { + Enabled -eq $true -and + PasswordNeverExpires -eq $false -and + PasswordLastSet -gt 0 + } + Properties = @( + 'Name' + 'EmailAddress' + 'msDS-UserPasswordExpiryTimeComputed' + 'UserPrincipalName' + ) + SearchBase = $LDAPdistinguishedName +} + +$SelectionProperties = @( + "Name" + "UserPrincipalName" + "EmailAddress" + @{ + Name = 'PasswordExpiry' + Expression = { + [datetime]::FromFileTime($_."msDS-UserPasswordExpiryTimeComputed").ToLongDateString() + } + } +) + +$Users = Get-ADUser @QueryParameters | Select-Object -Property $SelectionProperties +``` + +To see the result, call the `$Users` variable. + +```powershell-console +Name UserPrincipalName EmailAddress PasswordExpiry +---- ----------------- ------------ -------------- +test test@contoso.com test1@contoso.com Monday, August 15, 2022 +User1 User1@contoso.com User1@contoso.com Tuesday, October 18, 2022 +User2 User2@contoso.com User2@contoso.com Sunday, August 7, 2022 +. +. +. +Output trimmed +``` + +Later on a `ForEach-Object` loop goes through the users, and if any user **msDS-UserPasswordExpiryTimeComputed** matches the date in the `$DaysToSendWarning`, then the script sends them the chat message. +More to come in the full script. + +## Using Microsoft Graph to Create a Teams Chat Session + +[Microsoft Graph API documentation](https://docs.microsoft.com/graph/) is the best start with anything related to Graph API. + +We need to connect to the Graph API and use the following permission in the scope. + +```powershell +$Scope=@('Chat.Create','Chat.ReadWrite','User.Read','User.Read.All') +Connect-MgGraph -Scopes $Scope +``` + +First, start a chat session. The chat session contains a list of all the parties involved in the chat session. Also, it will provide a unique ID representing the communication between all the parties involved in the chat. + +```powershell-console +$NewChatIDParam = @{ + ChatType = "oneOnOne" + Members = @( + @{ + "@odata.type" = "#microsoft.graph.aadUserConversationMember" + Roles = @( + "owner" + ) + "User@odata.bind" = "https://graph.microsoft.com/v1.0/users('*XXXXXXXXXX*')" + } + @{ + "@odata.type" = "#microsoft.graph.aadUserConversationMember" + Roles = @( + "owner" + ) + "User@odata.bind" = "https://graph.microsoft.com/v1.0/users('*XXXXXXXXXX*')" + } + ) +} + +$ChatSessionID = New-MgChat -BodyParameter $NewChatIDParam + + +PS> $ChatSessionID.id +19:b980153c-9129-9129-9129-fb57a348d4d3_eafa5e65-9129-4e69-4e69-19f3fe6@unq.gbl.spaces + +``` + +Replace each `*XXXXXXXXXX*` with a user ID who is participating in the chat. It doesn't matter if the sender or the recipient is first. it's a two-way communication bridge. +But that the caller user id must be one of the members specified in the request body. + +You can get the user id by running `(Get-MgUser -userID user1@contoso.com).id` + +Read more about the parameters in the chat session from the [Create chat](https://docs.microsoft.com/graph/api/chat-post?view=graph-rest-beta&tabs=http). + +Executing the example above returns a long ID. The chat session ID must be used between these parties specified in the chat body. + +Running the example above again and again returns the same chat session id if the chat session already exists. +So no need to go through all the chat sessions to seek a certain chat conversation id. + +## Using Microsoft Graph to Send a Teams Chat Message + +As for now, we have the chat session id. +We can send a message with a line of code. + +```powershell +New-MgChatMessage -ChatId $ChatSessionID.id -Body @{Content ='<strong>Hello, I am PowerShell</strong>';ContentType='html'} +``` + +Looking to know more about `New-MgChatMessage` parameters, take a look at [Send chatMessage in channel or a chat](https://docs.microsoft.com/graph/api/chatmessage-post?view=graph-rest-beta&tabs=http) and also [Send HTML Teams Message Using PowerShell Graph](https://www.powershellcenter.com/2022/07/15/new-mgchat/). + +## Full Script to send notification + +So now we know the basics. Let's build it all together. + +There is no need to modify anything except the **$DaysToSendWarning** variable. +Set it to the number of days you want. +Everything else should be fine with no issues. + +You might need to consent and accept the new permission after connecting using the `Connect-MgGraph`. + +```powershell +Import-Module ActiveDirectory +Import-Module Microsoft.Graph.Teams + +$Scope = @( + 'Chat.Create' + 'Chat.ReadWrite' + 'User.Read' + 'User.Read.All' +) +Connect-MgGraph -Scopes $Scope + +$DaysToSendWarning = 7 + +#Find accounts that are enabled and have expiring passwords +$QueryParameters = @{ + Filter = { + Enabled -eq $true -and + PasswordNeverExpires -eq $false -and + PasswordLastSet -gt 0 + } + Properties = @( + 'Name' + 'EmailAddress' + 'msDS-UserPasswordExpiryTimeComputed' + 'UserPrincipalName' + ) + SearchBase = $LDAPdistinguishedName +} + +$SelectionProperties = @( + "Name" + "UserPrincipalName" + "EmailAddress" + @{ + Name = 'PasswordExpiry' + Expression = { + [datetime]::FromFileTime($_."msDS-UserPasswordExpiryTimeComputed").ToLongDateString() + } + } +) + +$Users = Get-ADUser @QueryParameters | Select-Object -Property $SelectionProperties + +foreach ($User in $Users) { + $RecpID = Get-MgUser -UserId $User.UserPrincipalName -ErrorAction Stop + if ($User.PasswordExpiry -eq $DaysToSendWarning) { + $NewChatIDParam = @{ + ChatType = "oneOnOne" + Members = @( + @{ + "@odata.type" = "#microsoft.graph.aadUserConversationMember" + Roles = @( + "owner" + ) + "User@odata.bind" = "https://graph.microsoft.com/v1.0/users('"+(get-mguser -userid (Get-MgContext).account).id +"')" + } + @{ + "@odata.type" = "#microsoft.graph.aadUserConversationMember" + Roles = @( + "owner" + ) + "User@odata.bind" = "https://graph.microsoft.com/v1.0/users('"+$RecpID.id +"')" + } + ) + } + + $ChatSessionID = New-MgChat -BodyParameter $NewChatIDParam + + Write-Host "Sending Message to $($RecpID.Mail)" -ForegroundColor Green + + try { + #### Sending The Message + $Body = @{ + ContentType = 'html' + Content = @" + Hello $($RecpID.DisplayName)<br> + Your password will expire in $($DaysToSendWarning), Please follow <Strong><a href='www.office.com'>the instruction here to update it</a> </Strong> <BR> + Thanks for your attention +"@ + } + + New-MgChatMessage -ChatId $ChatSessionID.ID -Body $Body -Importance Urgent + } catch{ + Write-Host $_.Exception.Message + } + } +} +``` + +## Conclusion + +This post shows how to send a basic HTML message, but there is still a lot. +Take a look at the Graph API documentation to know how to receive, read and have a wider control of not only Teams but also other Microsoft cloud services. + +It's ok to feel a bit lost about all these hashtables, arrays, and new things and the structure. No need to memorize it. just open the Graph API documentation, guiding you straight to the point. + +Lets me know if you try it; how did it go :) diff --git a/Posts/2022/07/ReadCMStatusMessages.md b/Posts/2022/07/ReadCMStatusMessages.md new file mode 100644 index 00000000..d8721b9f --- /dev/null +++ b/Posts/2022/07/ReadCMStatusMessages.md @@ -0,0 +1,375 @@ +--- +post_title: Reading Configuration Manager Status Messages With PowerShell +username: francisconabas +categories: PowerShell +tags: SCCM, MECM, Status Message, Config Manager +summary: This post's intent is to show how to read Configuration Manager status messages using WMI and Win32 API function FormatMessage. +--- + +**Q:** I can read Configuration Manager status messages using the _Monitoring_ tab. Can I do it +using PowerShell? + +**A:** Yes you can! We can accomplish this using SQL/WQL queries, plus the Win32 function +FormatMessage. + +## Better understanding Status Messages + +Before we get our hands dirty we need to understand how the Configuration Manager assembles these +messages and why we can't just query it from some table, view or WMI class. + +To avoid storage or performance issues and to provide better standardization, the Config Manager +stores only message's key information (and the ones who change from message to message), and uses a +Win32 function called FormatMessage together with a DLL to assembly and display the full message. + +At first, it seems intimidating, specially with the whole Win32 function thing, but it's actually +pretty simple. Let's take a look on one of these messages, so we can visualize what we want to +accomplish. + +``` +Distribution Manager failed to connect to the distribution point ["Display=\\\\CMGRDP1.contoso.com\\"]MSWNET:["SMS_SITE=PS1"]\\\\CMGRDP1.contoso.com\\. Check your network and firewall settings. +``` + +This message states a failed content distribution to a Distribution Point. If we remove the part of +the message containing the DP information, +`["Display=\\\\CMGRDP1.contoso.com\\"]MSWNET:["SMS_SITE=PS1"]\\\\CMGRDP1.contoso.com\\`, we end up with a +standard message that can be used every time this problem occurs. + +## Querying useful information + +Now that we have an overview of the Status Message structure, let's gather the information available +on the Config Manager database. For the purpose of this post, we will use failed distribution +messages, like the one we saw above. + +- The WMI classes that store Status Message information interesting for us are **SMS_StatusMessage** + and **SMS_StatMsgModuleNames**. +- For content distribution status we will use the **SMS_DistributionDPStatus** class. +- The SQL views for these classes are **v_StatusMessage**, **v_StatMsgModuleNames** and + **vSMS_distributionDPStatus** respectively. +- For performance sake and the SQL language accepting more complex queries we are going to use it + for our exercise. This SQL query should return all packages from our Distribution Point which the + status is not _Success_ or _InProgress_ + +```sql +SELECT * +FROM vSMS_DistributionDPStatus +WHERE [Name] = 'CMGRDP1.contoso.com' + AND MessageState NOT IN (1,2) +``` + +On the result, we are interested on some key columns: **MessageID**, **LastStatusID**, +**MessageSeverity** and the **InsString(n)**. + +- The **MessageID** and **MessageSeverity** we will use with the **FormatMessage** function. +- The **LastStatusID** we will use to join with the other views, who name this column **RecordID**. +- And perhaps the more interesting ones, the **InsString(n)** columns. + +These columns, **InsString1**, **InsString2**, **InsString3**, ..., **InsString10** contain the +custom part of the message. Let's look at one row of the above query shall we? + + +| ID<sup>1</sup> | MessageID | LastStatusID | MessageSeverity | InsString1<sup>2</sup> | InsString2 | +| :------------- | :-------- | :----------------- | :-------------- | :------------------------------------------------------------------------------ | :--------- | +| 47365 | 2391 | 216172782348300122 | -1073741824 | ["Display=\\\\CMGRDP1.contoso.com\\"]MSWNET:["SMS_SITE=PS1"]\\\\CMGRDP1.contoso.com\\ | | + +- <sup>1</sup> The **ID** column is to help us to identify this specific message later. +- <sup>2</sup> The other **InsString** columns are null + +Won't you look at that! The info on **InsString1** is exactly the custom part of our message! Let's +join the other views, and we will have all the information needed to proceed. We are also including +information from **v_Package**, or **SMS_Package** on WMI, to make the end result more meaningful. + +```sql +SELECT + pkg.Name + ,pkg.PackageID + ,dps.LastUpdateDate + ,stm.ModuleName + ,smn.MsgDLLName + ,dps.MessageID + ,CASE + WHEN dps.MessageSeverity = '1073741824' THEN '1073741824' --Informational + WHEN dps.MessageSeverity = '-2147483648' THEN '2147483648' --Warning + WHEN dps.MessageSeverity = '-1073741824' THEN '3221225472' --Error + END AS 'SeverityCode' + ,dps.InsString1 + ,dps.InsString2 + ,dps.InsString3 + ,dps.InsString4 + ,dps.InsString5 + ,dps.InsString6 + ,dps.InsString7 + ,dps.InsString8 + ,dps.InsString9 + ,dps.InsString10 +FROM vSMS_distributionDPStatus AS dps +LEFT JOIN v_StatusMessage AS stm ON stm.RecordID = dps.LastStatusID +LEFT JOIN v_StatMsgModuleNames AS smn ON smn.ModuleName = stm.ModuleName +LEFT JOIN v_Package AS pkg ON pkg.PackageID = dps.PackageID +WHERE dps.MessageState NOT IN (1,2) + AND dps.ID = '47365' +``` + +We are using the **ID** from the previous query to stick to our result. Removing this condition +should bring all package distribution failure for that site. + +The *Case* statement is necessary because the Message Severity is actually hexadecimal, thus: + +```powershell-console +PS C:\\> '{0:X}' -f -1073741824 +C0000000 +PS C:\\> '{0:X}' -f 3221225472 +C0000000 +PS C:\\> +PS C:\\> '{0:X}' -f -2147483648 +80000000 +PS C:\\> '{0:X}' -f 2147483648 +80000000 +PS C:\\> +``` + +Let's see what the result of this query looks like. + +- Name : Visual Studio 2019 Professional +- PackageID : PS100095 +- LastUpdateDate : 6/16/2022 3:49:26 AM +- ModuleName : SMS Server +- MsgDLLName : srvmsgs.dll +- MessageID : 2391 +- SeverityCode : 3221225472 +- InsString1 : ["Display=\\\\CMGRDP1.contoso.com\\"]MSWNET:["SMS_SITE=PS1"]\\\\CMGRDP1.contoso.com\\ +- InsString2 : +- InsString3 : +- InsString4 : +- InsString5 : +- InsString6 : +- InsString7 : +- InsString8 : +- InsString9 : +- InsString10 : + +As you can see, we have additional information here, especially **ModuleName** and **MsgDLLName**. +This DLL is the one we are going to use to format the message. + +## Formatting the message. Finally! + +To format our message to a readable format we will use the Configuration Manager SDK documentation, +which instruct us to use the Win32 API function *FormatMessage* together with the information we +just got. From the documentation: + +```cpp +// Get the module handle for the component's message DLL. This assumes the +// message DLL is loaded. If the DLL is not loaded, then load the DLL by using +// the Win32 API LoadLibrary. +hmodMessageDLL = GetModuleHandle(MsgDLLName); + +// The flags tell FormatMessage to allocate the memory needed for the message, +// to get the message text from a message DLL, and that the insertion strings are +// stored in an array, instead of a variable length argument list. The last +// parameter, apInsertStrings, is the array of insertion strings returned by the +// query. +dwMsgLen = FormatMessage(FORMAT_MESSAGE_ALLOCATE_BUFFER | + FORMAT_MESSAGE_FROM_HMODULE | + FORMAT_MESSAGE_ARGUMENT_ARRAY, + hmodMessageDLL, + Severity | MessageID, + 0, + lpBuffer, + nSize, + apInsertStrings); + +// Free the memory after you use the message text. +LocalFree(lpBuffer); +``` + +Wait a second... this is... C++? How am I supposed to call this function with PowerShell? + +We will borrow a platform from .NET called **PlatformInvoke** or ***Pinvoke*** for short. Combining +this through the namespace **System.Runtime.InteropServices** and importing as a type in PowerShell +using `Add-Type` will do the trick. + +> Disclaimer: Using Pinvoke to invoke unmanaged code is another beast in on itself and is beyond the +> scope of this post, however is lot's of fun! I'll leave a couple of links at the end to get you +> started. + +The first thing to do is to translate this C++ to C# so we can import into PowerShell. + +```csharp +namespace Win32Api +{ + using System; + using System.Text; + using System.Runtime.InteropServices; + + public class kernel32 + { + + [DllImport("kernel32.dll", CharSet=CharSet.Unicode, SetLastError=true)] + public static extern IntPtr GetModuleHandle( + string lpModuleName + ); + + [DllImport("kernel32.dll", CharSet=CharSet.Unicode, SetLastError=true)] + public static extern int FormatMessage( + uint dwFlags, + IntPtr lpSource, + uint dwMessageId, + uint dwLanguageId, + StringBuilder msgOut, + uint nSize, + string[] Arguments + ); + + [DllImport("kernel32", SetLastError=true, CharSet = CharSet.Unicode)] + public static extern IntPtr LoadLibrary( + string lpFileName + ); + + } + +} +``` + +Using `Add-Type` to import this namespace: + +```powershell +Add-Type -TypeDefinition @" +namespace Win32Api +{ + using System; + using System.Text; + using System.Runtime.InteropServices; + + public class kernel32 + { + + [DllImport("kernel32.dll", CharSet=CharSet.Unicode, SetLastError=true)] + public static extern IntPtr GetModuleHandle( + string lpModuleName + ); + + [DllImport("kernel32.dll", CharSet=CharSet.Unicode, SetLastError=true)] + public static extern int FormatMessage( + uint dwFlags, + IntPtr lpSource, + uint dwMessageId, + uint dwLanguageId, + StringBuilder msgOut, + uint nSize, + string[] Arguments + ); + + [DllImport("kernel32", SetLastError=true, CharSet = CharSet.Unicode)] + public static extern IntPtr LoadLibrary( + string lpFileName + ); + + } + +} +"@ +``` + +The SDK documentation lists 4 steps: + +1. Load the DLL with LoadLibrary. +2. Get a handle to this library with GetModuleHandle. +3. Call the FormatMessage function. +4. Free the memory after using the text with LocalFree + +Since we're calling this from PowerShell and the text will be loaded into a **StringBuilder** +object, the last step isn't necessary. The session will take care of the cleaning once we finish. + +So let's give it a go! + +```powershell +## Initializing the message and last error variables. Useful when processing lots of messages. +$lastError = $null +$message = $null + +## All modules location on the CM installation folder. +$smsMsgsPath = "$env:SystemDrive\\Program Files\\Microsoft Configuration Manager\\bin\\X64\\system32\\smsmsgs" +$moduleHandle = [Win32Api.kernel32]::GetModuleHandle("$smsMsgsPath\\srvmsgs.dll") ## The DLL From our query. + +## If the handle is zero, the module is not loaded. Checking to avoid loading the same DLL twice. +if ($moduleHandle -eq 0) { + [void][Win32Api.kernel32]::LoadLibrary("$smsMsgsPath\\srvmsgs.dll") + $moduleHandle = [Win32Api.kernel32]::GetModuleHandle("$smsMsgsPath\\srvmsgs.dll") +} + +$bufferSize = [int]16384 ## Buffer size for our output message. +## The StringBuilder object who will hold our message. +$bufferOutput = New-Object 'System.Text.StringBuilder' -ArgumentList $bufferSize + +$result = [Win32Api.kernel32]::FormatMessage( + 0x00000800 -bor 0x00000200 ## FORMAT_MESSAGE_FROM_HMODULE | FORMAT_MESSAGE_IGNORE_INSERTS + ,$moduleHandle + ,3221225472 -bor 2391 ## SeverityCode | MessageID + ,0 ## languageID. 0 = Default. + ,$bufferOutput + ,$bufferSize + ,$null ## Used to inject the InsStrings into the function. We'll process it later to avoid issues. +) + +## If the function returns zero, means a failure. Setting our $lastError variable to troubleshoot further. +if ($result -eq 0) { $lastError = [System.Runtime.InteropServices.Marshal]::GetLastWin32Error() } +``` + +At this point, if we did everything right the message should be stored on our **StringBuilder** +object. + +```powershell-console +PS C:\\> $result = [Win32Api.kernel32]::FormatMessage( +>> 0x00000800 -bor 0x00000200 ## FORMAT_MESSAGE_FROM_HMODULE | FORMAT_MESSAGE_IGNORE_INSERTS +>> ,$moduleHandle +>> ,3221225472 -bor 2391 ## SeverityCode | MessageID +>> ,0 ## languageID. 0 = Default. +>> ,$bufferOutput +>> ,$bufferSize +>> ,$null ## Used to inject the InsStrings into the function. We'll process it later to avoid issues. +>> ) +PS C:\\> $result +113 +PS C:\\> $bufferOutput.ToString() +%11Distribution Manager failed to connect to the distribution point %1. Check your network and firewall settings. +PS C:\\> +``` + +Eureka! We did it! + +And I bet you already know what that _%1_ means. ;). + +It's the location of our **InsString1**. + +So doing a little cleaning... + +_Assuming the result from our SQL query is stored on the variable `$fail`_: + +```powershell-console +PS C:\\> $message = $bufferOutput.ToString().Replace("%11","").Replace("%12","").Replace("%3%4%5%6%7%8%9%10","").Replace("%1",$fail.InsString1).Replace("%2",$fail.InsString2).Replace("%3",$fail.InsString3).Replace("%4",$fail.InsString4).Replace("%5",$fail.InsString5).Replace("%6",$fail.InsString6).Replace("%7",$fail.InsString7).Replace("%8",$fail.InsString8).Replace("%9",$fail.InsString9).Replace("%10",$fail.InsString10) +PS C:\\> +PS C:\\> $message +Distribution Manager failed to connect to the distribution point ["Display=\\\\CMGRDP1.contoso.com\\"]MSWNET:["SMS_SITE=PS1"]\\\\CMGRDP1.contoso.com\\. Check your network and firewall settings. +PS C:\\> +``` + +Now with the results of the query plus a beautifully formatted message you can store this into a +database or create your own reports and automations. Your imagination is the limit! + +## Conclusion + +Congratulations! You not only automated Configuration Manager Status Messages, but also called a +Win32 Native API function! + +I hope you had as much fun trying this as me writing it. + +Thank you very much, and I see you on the next trip! + +## Useful links + +- [Configuration Manager API Reference](https://docs.microsoft.com/mem/configmgr/develop/reference/configuration-manager-reference) +- [About Component Status Messages](https://docs.microsoft.com/mem/configmgr/develop/core/servers/manage/about-configuration-manager-component-status-messages) +- [FormatMessage Function winbase.h](https://docs.microsoft.com/windows/win32/api/winbase/nf-winbase-formatmessage) +- [LoadLibrary Function libloaderapi.h](https://docs.microsoft.com/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibrarya) +- [GetModuleHandle Function libloaderapi.h](https://docs.microsoft.com/windows/win32/api/libloaderapi/nf-libloaderapi-getmodulehandlea) +- [Platform Invoke (P/Invoke)](https://docs.microsoft.com/dotnet/standard/native-interop/pinvoke) +- [FormatMessage on pinvoke.net (With examples!)](https://www.pinvoke.net/default.aspx/kernel32.formatmessage) diff --git a/Posts/2022/07/cheat-sheet-console-experience.md b/Posts/2022/07/cheat-sheet-console-experience.md new file mode 100644 index 00000000..65ea5c5f --- /dev/null +++ b/Posts/2022/07/cheat-sheet-console-experience.md @@ -0,0 +1,230 @@ +--- +post_title: Cheat Sheet - Console Experience +username: frweinma +categories: PowerShell +tags: PowerShell, Preference Variables, Convenience, Tab Expansion +summary: How to have more control of preferences in functions and the role of modules on inheritance. +--- +PowerShell can take some getting used to. +Especially if you come at it from a different Shell and don't see any way to get your good old experience back. +However, hidden behind that plain white on blue shell, there is actually a wide range of customization options that help make your life less painful. +See below for the most commonly appreciated options. + +## Tab Completion + +The classic complaint we hear is that in Windows, Tab Completion is so much less helpful than for example in Bash. +This is mostly because <kdb>CTRL</kdb>+<kdb>Space</kdb> is hard to discover, unless somebody shows you. +Same menu choice as in Bash, but you can pick your preferred option using the arrow keys and each option may come with some documentation: + +![Console excerpt, showing a menu of parameter options for dir](./tab-completion.png) + +> Use whenever you would use the Tab Key + +## Key Bindings + +There are a few keybindings that come in handy to know: + +|Keybinding|Function| +|---|---| +|<kdb>Ctrl</kdb>+<kdb>Space</kdb>|Tab Menu| +|<kdb>Ctrl</kdb>+<kdb>r</kdb>|Search in your input history| +|<kdb>Ctrl</kdb>+<kdb>a</kdb>|Select everything in your current input/command line| +|<kdb>Ctrl</kdb>+<kdb>c</kdb>|Copy everything currently selected in your input/command line to your clipboard| +|<kdb>Ctrl</kdb>+<kdb>v</kdb>|Paste your clipboard into the current input/command line| +|<kdb>Shift</kdb>+<kdb>Enter</kdb>|Type multiline text in your console without executing the command| + +Specifically, it is important to get used to not pasting with right-click - by using <kdb>Ctrl</kdb>+<kdb>v</kdb> instead, you get a single input history for multiple lines, you can preview your input before sending it (helps with those artifacts you get when pasting from Teams) and you stop accidentally overwriting your clipboard by selecting something in the console window. + +Also, with right-click, you sometimes get the wrong order. + +Oh, and you can [define your own keybindings](https://github.com/PowerShell/PSReadLine/blob/master/PSReadLine/SamplePSReadLineProfile.ps1) if you want to. +No need to accept the defaults. + +## Packages + +There are plenty of PowerShell packages out there that can make console life a lot less painful. +Use `Find-Module` to search for them and `Install-Module` to install them. Example: + +```powershell +Find-Module *SQL* +Install-Module Powerline +``` + +Looking for a command but don't know the module it is from? + +```powershell +Find-Module -Command Write-PSFMessage +``` + +## Profile / Start Script + +The key to ultimate customization is to have a way to define code that runs on each console start without requiring manual action. +Now if only there were a way to do that in PowerShell ... + +```powershell +$profile +``` + +Yeah, that simple. As long as that file exists, it will be run. + +```powershell +notepad $profile +``` + +Add code, save, and you are good to go. + +> There are different profile files per application running PowerShell - VSCode has a different one than pwsh.exe than powershell.exe. +> Make sure you edit the file you meant to edit. +> Or update the global profile for all applications: `$profile.CurrentUserAllHosts` + +## PowerShell 7 / PowerShell Core + +There's Windows PowerShell, which comes installed by default on any Windows. +But there's also a [cool version you have to first install](https://aka.ms/powershell-release?tag=stable). +It adds great convenience, better performance and the ability to actually like using Visual Studio Code with PowerShell. +You can grab it via a wide variety of sources, such as the Microsoft Store, Github or your preferred package manager. + +You can also install it on MacOS or Linux. + +You should do so, it's awesome. + +## Prompt + +Want to customize your prompt to be more colorful / fancy / whatever else you want it to do? + +Well, all you need to do is override the function named `prompt` and put it in your profile and that's that. +Don't know how or want to borrow from others to make your life easier? + +Give [Powerline (for PowerShell)](https://github.com/Jaykul/PowerLine) a chance. +They have some [fancy examples](https://github.com/Jaykul/PowerLine/tree/master/Source/Examples) as well! + +![A colorful command prompt](./prompt.png) + +## Dynamic Tab Completion + +With the previous notes on Tab Completion, you already saw how to get better tab completion. +But PowerShell is still not reading your mind when it comes to the values provided - if a command doesn't offer it, you're out of luck. +Right? + +Well no, there's tools to fix that. +There are some options, but the simple-most is probably from the [PSFramework project](https://psframework.org/documentation/documents/psframework/tab-completion.html). +To install it, run this line (once): + +```powershell +Install-Module PSFramework +``` + +Then you can add the magic to your profile. +Here is a quick example on how to add values to the "-Tenant" parameter on "Set-AzContext": + +```powershell +Register-PSFTeppScriptblock -Name AZ.Tenant -ScriptBlock { + (Get-AZTenant).DefaultDomain +} +Register-PSFTeppArgumentCompleter -Command Set-AZContext -Parameter Tenant -Name AZ.Tenant +``` + +## The Clipboard + +The clipboard is always a handy tool to interactively cross over between applications. +Now if only there were commands in PowerShell to do so ... + +```powershell +Get-Clipboard +Set-Clipboard +``` + +And since we're all about being lazy, there's aliases for that: "gcb" and "scb". +On that note, if you want to paste multiple columns into Excel, you want to use the tab delimiter. +Don't do manual labor though, here's the easy way to get data ready to paste to Excel: + +```powershell +dir | ConvertTo-Csv -Delimiter "`t" | scb +``` + +That `ConvertTo-Csv` is way too much text though. Wouldn't it be nice to make that shorter? + +## Aliases + +In PowerShell, there is an easy way to be lazy: Aliases. +Use their power to abbreviate your commonly used commands. +Then put it in your `$profile` so you don't have to remember to add them. + +```powershell +Set-Alias ctc ConvertTo-Csv +``` + +And now that previous line can be shortened to: + +```powershell +dir | ctc -d "`t" | scb +``` + +Where is that "-d" coming from though? + +## Short Parameter Names + +You know all these commandline tools that have a long and a short notation for their parameters? +Like where you can either specify "--help" or "-h"? + +Well, PowerShell takes that a step further: +You only need to type enough of the parameter name to uniquely identify it. + +Using the example above with `ConvertTo-Csv`, there is only a single parameter that starts with "D", so that's enough to specify it. + +> Actually, there is a common parameter named "Debug", but those don't count here. + +## More Tools to improve the Console Experience + +A lot more tools have been created to help being lazy & comfortable than could possibly all be listed, but here a few more projects out there that can help make console life more comfortable: + +|Tool|Description| +|---|---| +|[AZ.Tools.Predictor](https://docs.microsoft.com/powershell/azure/az-predictor)|Predictive intellisense for the AZ modules| +|[TabExpansionPlusPlus](https://www.powershellgallery.com/packages/TabExpansionPlusPlus/1.2)|Adds tab completion for classic commandline tools such as ROBOCOPY| +|[PSUtil](https://www.powershellgallery.com/packages/PSUtil)|Adds keybindings, aliases and other conveniences| +|[oh-my-posh](https://ohmyposh.dev/)|Transform your prompt, alternative to the Powerline module shown above| +|[posh-git](https://github.com/dahlbyk/posh-git)|Add git integration to your prompt| + +## Paths, Explorer & PowerShell + +Often enough you want to interact with the file system across applications: + ++ Got the explorer open and want to start PowerShell in that path? ++ Just created an output file and want to open it? ++ Open the explorer in the current path? + +For all of that there are convenient options. +From within the shell, `Invoke-Item` or its alias `ii` allow you to open a path in its default application: + +```powershell +ii .\report.csv # Probably Excel +ii . # Current path in Explorer +``` + +The other way around works just as convenient. +In the Windows Explorer, just type `pwsh.exe` (or `powershell.exe`, if you didn't upgrade): + +![The Windows Explorer address bar, with the path replaced with "pwsh.exe"](./explorer.png) + +## Inspecting Output + +When you run a command, often enough you get some nice table, that is easy to read, but kind of lacking in data: + +![Results of a dir command in a neat table](./format-table.png) + +Fortunately, by piping to `FL *`, you get to see everything (even if it is a bit much): + +![Results of a dir command, with a long list of properties, hiding nothing](./format-list.png) + +## Concluding + +> "I designed PowerShell to optimize the user, not the code" +> +> -Jeffrey Snover, inventor of the PowerShell + +PowerShell allows us to optimize the way we work in the console, it is designed to help us automate and make problems go away. +So why do I see so many people who don't apply that same perspective to their own, personal console environment? +Go ahead and settle in in your console ... or face the charge of being insufficiently lazy! + +:) diff --git a/Posts/2022/07/explorer.png b/Posts/2022/07/explorer.png new file mode 100644 index 00000000..a3505164 Binary files /dev/null and b/Posts/2022/07/explorer.png differ diff --git a/Posts/2022/07/format-list.png b/Posts/2022/07/format-list.png new file mode 100644 index 00000000..5f96f463 Binary files /dev/null and b/Posts/2022/07/format-list.png differ diff --git a/Posts/2022/07/format-table.png b/Posts/2022/07/format-table.png new file mode 100644 index 00000000..365e1bab Binary files /dev/null and b/Posts/2022/07/format-table.png differ diff --git a/Posts/2022/07/prompt.png b/Posts/2022/07/prompt.png new file mode 100644 index 00000000..0c3b0848 Binary files /dev/null and b/Posts/2022/07/prompt.png differ diff --git a/Posts/2022/07/tab-completion.png b/Posts/2022/07/tab-completion.png new file mode 100644 index 00000000..b6f5e968 Binary files /dev/null and b/Posts/2022/07/tab-completion.png differ diff --git a/Posts/2022/08/Get-CimInstance_autoComplete.png b/Posts/2022/08/Get-CimInstance_autoComplete.png new file mode 100644 index 00000000..b8db4613 Binary files /dev/null and b/Posts/2022/08/Get-CimInstance_autoComplete.png differ diff --git a/Posts/2022/08/Invoke-CimMethod_autoComplete.png b/Posts/2022/08/Invoke-CimMethod_autoComplete.png new file mode 100644 index 00000000..126c3250 Binary files /dev/null and b/Posts/2022/08/Invoke-CimMethod_autoComplete.png differ diff --git a/Posts/2022/08/NamespaceManiputlation.png b/Posts/2022/08/NamespaceManiputlation.png new file mode 100644 index 00000000..5d3bf015 Binary files /dev/null and b/Posts/2022/08/NamespaceManiputlation.png differ diff --git a/Posts/2022/08/many-wmi-flavours.md b/Posts/2022/08/many-wmi-flavours.md new file mode 100644 index 00000000..72a45f59 --- /dev/null +++ b/Posts/2022/08/many-wmi-flavours.md @@ -0,0 +1,464 @@ +--- +post_title: The many flavours of WMI management +username: FranciscoNabas +categories: PowerShell +tags: PowerShell, WMI, .NET, COM, Windows Management +summary: Working with the different ways of managing WMI. +--- + +WMI is arguably one of the greatest tools a system administrator can have. You can manage Windows +workstations, interact with Microsoft products, like the Configuration Manager, monitor server's +resources and many more. Today, we are going to look at the different ways we can use WMI with +PowerShell. Hopefully, at the end, you will not have a favorite, but know what to use for each +occasion. + +## The Ways + +There are three tools for managing WMI I want to share with you. + +- The **System.Management** namespace. +- The WMI Scripting API. +- The CIM cmdlets. + +Wait, what about the WMI Cmdlets, like `Get-WmiObject`? There are two reasons we are not covering +these today. These commands are only available for Windows PowerShell, and you will come to learn +that the **System.Management** namespace is very similar. If you are still resisting trying +PowerShell 7, I could not recommend it enough. + +## The Procedure + +I want to cover tasks we face everyday while administering Windows devices. We will look at: + +- Querying. +- Calling a WMI Class method. +- Creating, Updating and Deleting a WMI Class Instance. +- Bonus: Creating, Populating and Deleting a custom WMI Class. + +I also want to show the pros and cons of each method, and where one stands out from the others. + +## The System.Management Namespace + +If I had to pick a favorite, it would be this one. Bringing an object-oriented "feel" to WMI, this +.NET namespace makes WMI management intuitive. Plus, if you are a C# developer, this will feel like +home. + +### Querying + +To perform a query, we need an instance of the **ManagementObjectSearcher** class. +There are three constructors worth looking at: + +- `ManagementObjectSearcher(String)` + - The simplest one. Creates a searcher object specifying the query string. +- `ManagementObjectSearcher(String, String)` + - Creates the object with the query and the scope. +- `ManagementObjectSearcher(ManagementScope, ObjectQuery)` + - The same as the previous one, but with instances of the objects instead of strings. This gives + you more options. + +Once we have the searcher, we call the **Get** method to return the **ManagementObjects**. + +```powershell +$query = "Select * From Win32_Process Where Name = 'powershell.exe'" +$searcher = [wmisearcher]($query) +$result = $searcher.Get() +``` + +The `$result` variable holds an instance of the **ManagementObjectCollection** class. +This collection contains all the **Win32_Process** instances in the form of **ManagementObjects**. + +```powershell +$result = $searcher.Get() +$result | Format-Table -Property ProcessId, Name, ExecutablePath -AutoSize + +```Output +ProcessId Name ExecutablePath +--------- ---- -------------- + 4116 powershell.exe C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe +``` + +This is how it looks like using the second and third constructors. + +```powershell +$query = "Select * From Win32_Process Where Name = 'powershell.exe'" +$scope = 'root\cimv2' +$searcher = [wmisearcher]::new($scope, $query) +$result = $searcher.Get() + +# Or + +$query = [System.Management.ObjectQuery]::new("Select * From Win32_Process Where Name = 'powershell.exe'") +$scope = [System.Management.ManagementScope]::new('root\cimv2') +$scope.Connect() +$searcher = [System.Management.ManagementObjectSearcher]::new($scope, $query) +$result = $searcher.Get() +``` + +### Calling a WMI Method + +We could either call a method on the **ManagementObject** resultant from our query operation, like +**Terminate**, or call a method on the WMI Class object. Let's create a new process using the +**Create** method. + +```powershell +$commandLine = 'powershell.exe -ExecutionPolicy Bypass -Command "Write-Output ''Howdy! From WMI!''; Read-Host"' +$processClass = [wmiclass]'Win32_Process' +# The parameters are: CommandLine, CurrentDirectory and ProcessStartupInformation. +$processClass.Create($commandLine, $null, $null) +``` + +If the method succeeds, you should be presented with a PowerShell console, and the **Output Parameters**: + +```Output +__GENUS : 2 +__CLASS : __PARAMETERS +__SUPERCLASS : +__DYNASTY : __PARAMETERS +__RELPATH : +__PROPERTY_COUNT : 2 +__PROPERTY_COUNT : 2 +__DERIVATION : {} +__SERVER : +__NAMESPACE : +__PATH : +ProcessId : 11896 +ReturnValue : 0 +PSComputerName : +``` + +### Creating, Updating and Deleting a WMI Class Instance + +We are going to use the `ManagementClass.CreateInstance()` method to create a new instance of the +**SMS_Collection** class and **Put** to save it to the namespace. + +```powershell +$collection = ([wmiclass]'root\SMS\site_PS1:SMS_Collection').CreateInstance() +$collection.Name = 'AwesomeDeviceCollection' +$collection.LimitingCollectionID = 'PS1000042' +$collection.Put() +# The Get() method updates the $collection object with the new +# property values populated by the Config Manager. +$collection.Get() +``` + +Updating and deleting. + +```powershell +$collection = [wmiclass]"root\SMS\site_PS1:SMS_Collection.CollectionID='PS1000043'" +$collection.Name = 'AwesomeDeviceCollection_NewName' +$collection.Put() + +# Deleting + +$collection.Delete() +``` + +## The WMI Scripting API + +The WMI Scripting API is nothing more than the exposure of the WMI COM interfaces through a Runtime +Callable Wrapper. Or WMI COM Object, for short. This method of managing WMI is not as straight +forward as the **System.Management** namespace, but its implementation gives great flexibility. + +### Querying + +To start, we need to instantiate a **SWbemLocator** object, which will be the interface to the other +objects, and obtain a **SWbemServices** object, by connecting to the server. + +```powershell +$locator = New-Object -ComObject 'WbemScripting.SWbemLocator' +$services = $locator.ConnectServer() +``` + +Then, we use the **ExecQuery** method, from the **SWbemServices** object to perform our query. This +method returns a **SWbemObjectSet**, which is a collection of **SWbemObjects**. Its properties are +under the **Properties_** property. + +```powershell +$result = $services.ExecQuery("Select * From Win32_Process Where Name = 'powershell.exe'") +$object = $result | Select-Object -First 1 +$value = $object.Properties_['ProcessId'].Value +``` + +### Calling a WMI Method + +First, we need to create an instance of the **__Properties** class, which holds the input parameters +for the **Create** method. Then, we use the `SWbemServices.ExecMethod()` method to call **Create**. + +```powershell +$commandLine = 'powershell.exe -ExecutionPolicy Bypass -Command "Write-Output ''Howdy! From WMI!''; Read-Host"' +$parameters = $object.Methods_['Create'].InParameters.SpawnInstance_() +$parameters.Properties_['CommandLine'].Value = $commandLine + +$output = $services.ExecMethod('Win32_Process', 'Create', $parameters) +``` + +The `$output` variable contains a **SWbemObject**, which is an instance of the **Output Parameters** +property class. + +```powershell +$services.ExecMethod('Win32_Process', 'Create', $parameters) +``` + +```Output +Value : 16172 +Name : ProcessId +IsLocal : True +Origin : __PARAMETERS +CIMType : 19 +Qualifiers_ : System.__ComObject +IsArray : False + +Value : 0 +Name : ReturnValue +IsLocal : True +Origin : __PARAMETERS +CIMType : 19 +Qualifiers_ : System.__ComObject +IsArray : False +``` + +### Creating, Updating and Deleting a WMI Class Instance + +Let's replicate our last example using the Scripting API. + +```powershell +$collection = $services.Get('\\.\root\SMS\site_PS1:SMS_Collection').SpawnInstance_() +$collection.Properties_['Name'].Value = 'AwesomeDeviceCollection' +$collection.Properties_['LimitingCollectionID'].Value = 'PS1000042' +$collection.Put_() +``` + +Updating and deleting. + +```powershell +$collection = $services.Get("\\.\root\SMS\site_PS1:SMS_Collection.CollectionID='PS1000043'") +$collection.Properties_['Name'].Value = 'AwesomeDeviceCollection_NewName' +$collection.Put_() + +# Deleting + +$collection.Delete_() +``` + +## The CIM Cmdlets + +When you just want to perform a WMI query to analyze data, and not necessarily interact with it, you +cannot beat the CIM Cmdlets. They are extremely fast, and provide unique tools like auto-complete +for class and namespace names and easy class retrival with `Get-CimClass`. + +### Querying + +Performing queries with the CIM Cmdlets is very pleasant. One line does it all. + +```powershell +$result = Get-CimInstance -Query "Select * From Win32_Process Where Name = 'powershell.exe'" +``` + +The parameters are very similar to the `Get-WmiObject` ones, and can be used as follows. + +```powershell +$result = Get-CimInstance -ClassName 'Win32_Process' -Filter "Name = 'powershell.exe'" +``` + +The _auto-complete_ feature in Visual Studio Code. + +![Auto-Complete with CIM](Get-CimInstance_autoComplete.png) + +### Calling a WMI Method + +The CIM Cmdlets introduces a unique way of calling WMI Methods. The results of a CIM query are +called **CimInstances**, and you cannot call instance methods like you would with the other two +options. Instead, you call another Cmdlet called `Invoke-CimMethod`. + +```powershell +$commandLine = 'powershell.exe -ExecutionPolicy Bypass -Command "Write-Output ''Howdy! From WMI!''; Read-Host"' +$result = Get-CimClass -ClassName 'Win32_Process' +$params = @{ + MethodName = 'Create' + Arguments = @{ + CommandLine = $commandLine + } +} +$output = $result | Invoke-CimMethod @params + +# Or +$params = @{ + ClassName = 'Win32_Process' + MethodName = 'Create' + Arguments = @{ + CommandLine = $commandLine + } +} +$output = Invoke-CimMethod @params +``` + +And the result: + +```powershell +Invoke-CimMethod -ClassName 'Win32_Process' -MethodName 'Create' -Arguments @{ CommandLine = $commandLine } +``` + +```Output +ProcessId ReturnValue PSComputerName +--------- ----------- -------------- + 14932 0 +``` + +Have a hard time remembering parameters? Me too! Luckily _auto-complete_ also works with them. + +![Auto-Complete with method parameters](Invoke-CimMethod_autoComplete.png) + +### Creating, Updating and Deleting a WMI Class Instance + +If you used the old WMI Cmdlets before, this will look familiar. + +```powershell +$params = @{ + Namespace = 'root\SMS\site_PS1' + ClassName = 'SMS_Collection' + Property = @{ + Name = 'AwesomeDeviceCollection' + LimitingCollectionID = 'PS1000042' + } +} +$collection = New-CimInstance @params +``` + +Updating and deleting. + +```powershell +$params = @{ + Namespace = 'root\SMS\site_PS1' + Query = "Select * From SMS_Collection Where Name = 'AwesomeDeviceCollection'" + Property = @{ + Name = 'AwesomeDeviceCollection_NewName' + } +} +Set-CimInstance @params + +#Or + +$params = @{ + Namespace = 'root\SMS\site_PS1' + Query = "Select * From SMS_Collection Where Name = 'AwesomeDeviceCollection'" +} +$collection = Get-CimInstance @params +$collection | Set-CimInstance -Property @{ + Name = 'AwesomeDeviceCollection_NewName' +} + +#Deleting + +$params = @{ + Namespace = 'root\SMS\site_PS1' + Query = "Select * From SMS_Collection Where Name = 'AwesomeDeviceCollection'" +} +$collection = Get-CimInstance @params +$collection | Remove-CimInstance +``` + +## Pros and Cons + +- The **System.Management** namespace is great for acquiring instances of objects or classes. The + aliases like `[wmi]` or `[wmiclass]` makes it easy to work with them, if you know their path. + Calling methods is also very intuitive. In the other hand, querying and doing more complex + operations can be time-consuming, and can involve more objects to keep track of. + +- Using the WMI Scripting API is great when you have to build whole scripts to manage WMI. Once you + have the **SWbemServices** object you can work with pretty much anything else. It is also + rewarding performance-wise, compared to the previous method, since you are working with the RCW + interfaces directly. The **System.Management** namespace will wrap these interfaces to provide + abstraction. But this comes at a cost. If you want to retrieve single objects or perform queries + to analyze data, this method can be a little annoying to work with. + +- The CIM cmdlets are number one on performance when querying multiple-object datasets. The + **CimInstance** object is great to work with, specially when combining with other known objects, + like the **PSCustomObject**. On the other hand, calling methods is not as straight forward as on + the previous methods. And to access methods like **Put**, **Get** or **Delete** can be + challenging. + +## Bonus + +Let's create our own Namespace under root, and implement a custom class! We will use the +**System.Management** namespace, but now you can use what you learn to implement this using the +other methods as well. + +```powershell +$namespace = ([wmiclass]'root:__Namespace').CreateInstance() +$namespace.Name = 'ScriptingBlogCoolNamespace' +$namespace.Put() +``` + +```Output +Path : \\.\root:__NAMESPACE.Name="ScriptingBlogCoolNamespace" +RelativePath : __NAMESPACE.Name="ScriptingBlogCoolNamespace" +Server : . +NamespacePath : root +ClassName : __NAMESPACE +IsClass : False +IsInstance : True +IsSingleton : False +``` + +```powershell +$class = [wmiclass]::new('root\ScriptingBlogCoolNamespace', '', $null) +$class['__Class'] = 'CustomClass' +$class.Qualifiers.Add('Static', $true) +$class.Properties.Add('Source', 'Custom class with .NET!') +$class.Properties.Add('PropertyKey', 1) +## You need a Key property, otherwise WMI wouldn't be able to assemble the path of a new instance. +$class.Properties['PropertyKey'].Qualifiers.Add('Key', $true) +$class.Put() +``` + +```Output +Path : \\.\root\ScriptingBlogCoolNamespace:CustomClass +RelativePath : CustomClass +Server : . +NamespacePath : root\ScriptingBlogCoolNamespace +ClassName : CustomClass +IsClass : True +IsInstance : False +IsSingleton : False +``` + +```powershell +$instance = ([wmiclass]'root\ScriptingBlogCoolNamespace:CustomClass').CreateInstance() +$instance.Source = 'CustomInstance!' +$instance.Put() +``` + +```Output +Path : \\.\root\ScriptingBlogCoolNamespace:CustomClass.PropertyKey=1 +RelativePath : CustomClass.PropertyKey=1 +Server : . +NamespacePath : root\ScriptingBlogCoolNamespace +ClassName : CustomClass +IsClass : False +IsInstance : True +IsSingleton : False +``` + +And just like that, we have our own Namespace, Class and Instance! +The results on **WmiExplorer**. + +![Namespace, Class and Instance](NamespaceManiputlation.png) + +## Conclusion + +If you made it to the end, hopefully now you have the right tool for the right job, regarding WMI. +There is no winner, all of them are good in specific situations. +What they all have in common is that they, together, will make you a better System Administrator. + +Thank you for following along on this journey, and I see you next time! + +## Useful links + +- [System.Management](https://docs.microsoft.com/en-us/dotnet/api/system.management?view=dotnet-plat-ext-6.0) +- [WMI Scripting API](https://docs.microsoft.com/en-us/windows/win32/wmisdk/scripting-api-for-wmi) +- [CIM Cmdlets](https://docs.microsoft.com/en-us/powershell/module/cimcmdlets/?view=powershell-7.2) +- [Runtime-Callable Wrapper](https://docs.microsoft.com/en-us/dotnet/standard/native-interop/runtime-callable-wrapper) +- [WMI Documentation](https://docs.microsoft.com/en-us/windows/win32/wmisdk/wmi-start-page) + +See what I am up to! + +[Github](https://github.com/FranciscoNabas) \ No newline at end of file diff --git a/Posts/2022/09/Registry-Monitor.md b/Posts/2022/09/Registry-Monitor.md new file mode 100644 index 00000000..5000c146 --- /dev/null +++ b/Posts/2022/09/Registry-Monitor.md @@ -0,0 +1,277 @@ +--- +post_title: PowerShell Registry Monitor +username: FranciscoNabas +categories: PowerShell +tags: PowerShell, Registry, Win32, COM, Windows Management +summary: How to set up a simple registry key monitor with PowerShell +--- + +Recently, while discussing work-related topics, a co-worker asked me if there is a way of monitoring +changes on a Windows registry key. I knew we can monitor files, with the +**System.IO.FileSystemWatcher** .NET class, but never heard of registry monitoring. Well, turns out +Windows provides an API for it, and with the help of Interop Services, we can call it from +PowerShell. + +## About tools + +To accomplish this, we will need to work with **Platform Invoke**, or _PinVoke_ for short. It +consists of a .NET library that wraps the native APIs to be called by managed .NET code. This +library comes with Windows, on the Global Assembly Cache, and also with PowerShell Core. + +In addition to that, we will work with a couple of Windows API functions, listed below: + +- **RegOpenKeyEx:** Responsible for opening a handle[^1] to the key. +- **RegNotifyChangeKeyValue:** Responsible for monitoring the key, and triggering an event when a + change happens. +- **CreateEvent:** Responsible for creating the event. +- **WaitForSingleObject:** This will monitor the event, and return a result based on the outcome. +- **RegCloseKey:** To close the handle to our registry key. +- **CloseHandle:** To close the handle to the event created. + +The last two commands are not mandatory, because the Interop Services will wrap the handles in +something called **Safe Handle**. This handle is released by the Garbage Collector at the end, but +it's not only a good practice, it creates the habit of monitoring object's lifecycles. If we are +looking into interoperating with Windows more often, we need to get used to how it manages memory, +to avoid unexpected behavior. + +If you want a series of posts based on **PinVoke** and interoperability, let me know in the +comments! + +## About definition + +If we want to leverage **System.Runtime.InteropServices**, we need to write part of our code in C#. +Don't get intimidated, C# and PowerShell are very similar, and it won't be hard at all. Let's start +by defining our functions. + +I will demonstrate step by step with **RegOpenKeyEx**, and the others will follow the same +procedure. From Microsoft's documentation page, the function definition looks like this: + +```cpp +LSTATUS RegOpenKeyExW( + [in] HKEY hKey, + [in, optional] LPCWSTR lpSubKey, + [in] DWORD ulOptions, + [in] REGSAM samDesired, + [out] PHKEY phkResult +); +``` + +Don't worry about the `W` at the end. Most of Windows functions have two versions, the **ANSI** +version, and **UNICODE** version. Functions terminated in `A` are **ANSI** compliant, the `W` ones +comply to **UNICODE**. If you call **RegOpenKeyEx**, Windows will route to one of the two. + +In order to represent this function with C#, we need to convert the parameter types. This process is +often called _marshalling_. We can interpret these types as follows: + +- `HKEY`: This represents a **handle**. Handles are a type of **Pointer**, so we can represent it as + **System.IntPtr**. Since memory addresses are numbers, this type is a special kind of integer. +- `LPCWSTR`: A pointer to a constant string with 16-bit Unicode characters. For us, a + **System.String**. +- `DWORD`: A 32-bit unsigned integer. In other words, a **System.UInt32**. +- `REGSAM`: A Registry Security Access Mask. We will talk about it in a bit. +- `PHKEY`: A pointer to a variable that will receive the opened key handle. We know that we can + represent pointers as **System.IntPtr**. +- `LSTATUS`: The function's return type. This maps to a **long**. We will represent it with + **System.Int**. + +The **REGSAM** data type is a list of definitions that will map Registry Key security to unsigned +integers, so we can represent it as a **System.Uint32**. We will be using the **KEY_NOTIFY** REGSAM, +which translates to `0x0010`. At the end, our function definition will look something like this: + +```csharp +[DllImport("Advapi32.dll", CharSet = CharSet.Unicode, SetLastError = true)] +public static extern int RegOpenKeyExW( + IntPtr hKey, + string lpSubKey, + uint ulOptions, + uint samDesired, + out IntPtr phkResult +); +``` + +The first line in square brackets is called **DllImport Attribute**. It's what tells PinVoke which DLL +contains the definition for **RegOpenKeyExW**. `CharSet = CharSet.Unicode` defines Unicode as our +encoding, and `SetLastError = true` will set the last error with the corresponding Win32 error, if +the function call fails. Setting the last error is crucial for debugging and troubleshooting these +function calls. + +Following the same approach, we write the full code: + +```csharp +using System; +using System.Runtime.InteropServices; + +namespace Win32 +{ + public class Regmon + { + [DllImport("Advapi32.dll", CharSet = CharSet.Unicode, SetLastError = true)] + public static extern int RegOpenKeyExW( + int hKey, + string lpSubKey, + int ulOptions, + uint samDesired, + out IntPtr phkResult + ); + + [DllImport("Advapi32.dll", CharSet = CharSet.Unicode, SetLastError = true)] + public static extern int RegNotifyChangeKeyValue( + IntPtr hKey, + bool bWatchSubtree, + int dwNotifyFilter, + IntPtr hEvent, + bool fAsynchronous + ); + + [DllImport("Advapi32.dll", CharSet = CharSet.Unicode, SetLastError = true)] + public static extern int RegCloseKey(IntPtr hKey); + + [DllImport("Advapi32.dll", CharSet = CharSet.Unicode, SetLastError = true)] + public static extern int CloseHandle(IntPtr hKey); + + [DllImport("kernel32.dll", CharSet = CharSet.Unicode, SetLastError = true)] + public static extern IntPtr CreateEventW( + int lpEventAttributes, + bool bManualReset, + bool bInitialState, + string lpName + ); + + [DllImport("kernel32.dll", CharSet = CharSet.Unicode, SetLastError = true)] + public static extern int WaitForSingleObject( + IntPtr hHandle, + int dwMilliseconds + ); + } +} +``` + +Originally, the parameter **lpEventAttributes** is from the **LPSECURITY_ATTRIBUTES**, which is a +structure. Since we are not going to use it, defining as **int** won't cause troubles. If we were to +use it, we could define **LPSECURITY_ATTRIBUTES**[^2]. + +## Writing the PowerShell code + +Now that all the paper work is done, we can write the PowerShell code that will use these functions. +To avoid filling your screen with repetitive code, I will represent the previous definition text as +`$signature`. You just have to create a string that will receive the C# code. I use here-strings: + +```powershell +$signature = @' + Your code goes here. +'@ +``` + +The final script looks like this: + +```powershell +using namespace System.Runtime.InteropServices + +[CmdletBinding()] +param ( + [Parameter(Mandatory)] + [string]$KeyPath, + + [Parameter()] + [string]$LogPath = "$PSScriptRoot\RegMon-$(Get-Date -Format 'yyyyMMdd-hhmmss').log", + + [Parameter()] + [int]$Timeout = 0xFFFFFFFF #INFINITE +) + +Add-Type -TypeDefinition $signature + +if (!(Test-Path -Path $KeyPath)) { throw "Registry key not found." } + +switch -Wildcard ((Get-Item $KeyPath).Name) { + 'HKEY_CLASSES_ROOT*' { $regdefault = 0x80000000 } + 'HKEY_CURRENT_USER*' { $regdefault = 0x80000001 } + 'HKEY_LOCAL_MACHINE*' { $regdefault = 0x80000002 } + 'HKEY_USERS*' { $regdefault = 0x80000003 } + Default { throw 'Unsuported hive.' } +} + +$handle = [IntPtr]::Zero +$result = [Win32.Regmon]::RegOpenKeyExW($regdefault, ($KeyPath -replace '^.*:\\'), 0, 0x0010, [ref]$handle) +$event = [Win32.Regmon]::CreateEventW($null, $true, $false, $null) + +<# +This will run indefinitely until it fails or reaches a timeout. +Adjust accordingly. +#> +:Outer while ($true) { + $result = [Win32.Regmon]::RegNotifyChangeKeyValue( + $handle, + $false, + 0x00000001L -bor #REG_NOTIFY_CHANGE_NAME + 0x00000002L -bor #REG_NOTIFY_CHANGE_ATTRIBUTES + 0x00000004L -bor #REG_NOTIFY_CHANGE_LAST_SET + 0x00000008L, #REG_NOTIFY_CHANGE_SECURITY + $event, + $true + ) + $wait = [Win32.Regmon]::WaitForSingleObject($event, $Timeout) + + switch ($wait) { + 0xFFFFFFFF { break Outer } #WAIT_FAILED + + 0x00000102L { #WAIT_TIMEOUT + $outp = 'Timeout reached.' + Write-Host $outp -ForegroundColor DarkGreen + Add-Content -FilePath $LogPath -Value $outp + break Outer + } + + 0 { #WAIT_OBJECT_0 ~> Change detected. + $outp = "Change triggered on the specified key. Timestamp: $(Get-Date -Format 'hh:mm:ss - dd/MM/yyyy')." + Write-Host $outp -ForegroundColor DarkGreen + Add-Content -FilePath $LogPath -Value $outp + } + } +} + +[Win32.Regmon]::CloseHandle($event) +[Win32.Regmon]::RegCloseKey($handle) +``` + +[alert type="note" heading="Note"] +When calling **RegOpenKeyExW** for the first time, we don't have the handle to the key yet, so we +specify which root key we want to use. The parameter **lpSubKey** is optional. When not specified, +the function will monitor the root key. +[/alert] + +## Caveats + +The **RegNotifyChangeKeyValue** is limited on what information it provides to the caller. If the +parameter **bWatchSubtree** is false, the function will monitor only the key specified. If this +parameter is true, the function monitors subtrees, but if an event is triggered, it will not inform +which key was modified. + +Is there a way of getting more information about Registry Events? Yes[^3], but this is a topic for +another post. + +## Conclusion + +I hope this post made calling Windows API Functions with PowerShell less intimidating. Once you get +used to Platform Invoke, you will need a bigger toolbox to store your new tools. + +Thank you for following along, once again, and I will see you next time! + +Useful links. + +- [Platform Invoke][pinvoke-docs] +- [A great resource of examples and how-tos for PinVoke][pinvoke-net] + +Want to test, or give suggestions on our **WindowsUtils** PowerShell module? Check out +[Windows Utils][windows-utils]. + +[See what I'm up to][github-profile]. + +[^1]: [Handles and Objects](https://docs.microsoft.com/en-us/windows/win32/sysinfo/handles-and-objects) +[^2]: [SECURITY_ATTRIBUTES structure](https://docs.microsoft.com/en-us/previous-versions/windows/desktop/legacy/aa379560(v=vs.85)) +[^3]: [Registry and Event Tracing](https://docs.microsoft.com/en-us/windows/win32/etw/registry) + +[github-profile]: https://github.com/FranciscoNabas/ +[pinvoke-docs]: https://docs.microsoft.com/en-us/dotnet/standard/native-interop/pinvoke +[pinvoke-net]: https://www.pinvoke.net/ +[windows-utils]: https://github.com/FranciscoNabas/WindowsUtils diff --git a/Posts/2023/01/Mastering-The-Steppable-Pipeline.md b/Posts/2023/01/Mastering-The-Steppable-Pipeline.md new file mode 100644 index 00000000..06378ccc --- /dev/null +++ b/Posts/2023/01/Mastering-The-Steppable-Pipeline.md @@ -0,0 +1,380 @@ +--- +post_title: Mastering the (steppable) pipeline +username: boderonald +categories: PowerShell +tags: PowerShell, Pipeline, Steppable +summary: The PowerShell pipeline explained from the beginning to the end. +--- + +# Mastering the (steppable) pipeline + +Before stepping into the *steppable* pipeline, it is essential that you have a good understanding of +how *and when* exactly items are processed by a [cmdlet][07] in the pipeline. The PowerShell +pipeline might just look like syntactical sugar but it is a lot more than that. In fact, it really +*acts* like a pipeline where each item flows through and is handled by each cmdlet one-at-a-time. In +comparison to the pipes in CMD, PowerShell streams *objects* through the pipeline rather than plain +text. + +## One-at-a-time `process` + +The following explanation describes the **one-at-a-time processing** section of the +[About pipelines][03] document. A good analogy of the pipeline is a physical assembly line where +each consecutive station on the line could be compared with a PowerShell cmdlet. At a specific +station and time, some something is done to one item while the next item is prepared at the prior +station. For example, at station 2 a component is soldered to the assembly while the next item is +being unpacked at station 1. Items iterate through the pipeline like: + +**Iteration: `n`** + +``` + item 3 --> item 2 --> item 1 +Station 1 | Station 2 | Station 3 +``` + +**Iteration: `n + 1`** + +``` + item 4 --> item 3 --> item 2 +Station 1 | Station 2 | Station 3 +``` + +Cmdlets act like stations in the assembly line, taken a simple example: + +```PowerShell +Get-Content .\Input.txt | Foreach-Object { $_ } | Set-Content .\Output.txt +``` + +In this example the `Foreach-Object { $_ }` cmdlet does nothing more than: + +* picking up each item from the pipeline that has been output by the prior cmdlet + `Get-Content .\Input.txt` +* placing it back on the pipeline as an input for the next cmdlet `Set-Content .\Output.txt`. + +To visualize the order of the items that go through the `Foreach-Object { $_ }` cmdlet you might use +the `Trace-Command` cmdlet but that might overwhelm you with data. Instead, using two simple +`ForEach-Object` (alias `%`) test commands show you exactly where your measure points are and what +goes in and come out the specific cmdlet in between. + +- `%{Write-Host 'In:' $_; $_ }` +- `%{Write-Host 'out:' $_; $_ }` + +Notice that `...; $_ }` in the end of the command will place the current item back on the pipeline. +In the following example, the cmdlet at the start of the pipeline (`Get-Content .\Input.txt`) has +been replaced with 4 hardcoded input items (`1,2,3,4`) and the cmdlet at the end of the pipeline +(`Set-Content .\Output.txt`) with `Out-Null` which simply purges the actual output of the pipeline +so that only the two test cmdlets produce an output. + +```PowerShell +1,2,3,4 | %{Write-Host 'In:' $_; $_ } | + Foreach-Object { $_ } | + %{Write-Host 'Out:' $_; $_ } | + Out-Null +``` + +This shows the following output: + +```Console +In: 1 +Out: 1 +In: 2 +Out: 2 +In: 3 +Out: 3 +In: 4 +Out: 4 +``` + +This proves that each item flows out of the pipeline (`Out: 1`) before the next item (`In: 2`) is +injected into it. As you can imagine, this conserves memory as there are only a few items in the +pipeline at any time. + +## Chocking the pipeline + +The previous section explains how a cmdlet would perform if correctly implemented for the middle of +a pipeline but there are a few statements that might "**choke**" the pipeline, meaning that the +items are no longer processed **one-at-the-time** but piled up in memory and eventually processed +**all-at-once**. This happens for: + +* **Assigning the pipeline to a variable**: + + ```PowerShell + $Content = Get-Content .\Input.txt | Foreach-Object { $_ } + $Content | Set-Content .\Output.txt + ``` + +* **Using parentheses**: + + ```PowerShell + (Get-Content .\Data.txt | Foreach-Object { $_ }) | Set-Content .\Data.txt + ``` + +* **Some cmdlets might choke the pipeline by design:** + + In general, a well defined cmdlet should write single records to the pipeline. See the + [Strongly Encouraged Development Guidelines][08] + article. + + Yet this is not always possible. Take, for example, the `Sort-Object` cmdlet, which is supposed to + sort an object collection. This might result is a new list where the last item ends up first. To + determine what item comes first, you must collect all items before they can be sorted. This is + visible from the simple test commands used before: + + ```PowerShell + 1,2,3,4 | %{Write-Host 'In:' $_; $_ } | Sort-Object | %{Write-Host 'Out:' $_; $_ } | Out-Null + ``` + + This shows the following output: + + ```Console + In: 1 + In: 2 + In: 3 + In: 4 + Out: 1 + Out: 2 + Out: 3 + Out: 4 + ``` + +In general, you should avoid chocking the pipeline, but their are few exceptions where it might be +required. For example, where you want to read and write back to the same file as in the previous +"using parenthesis" example. + +In a smooth pipeline, each item is processed one-at-the-time, meaning that `Get-Content` and +`Set-Content` are concurrently processing items in the pipeline. This causes the following error: + +> The process cannot access the file '.\Data.txt' because it is being used by another process. + +In this situation, chocking the pipeline and reading the complete file first avoids the error. + +### Heavy objects + +Objects in the PowerShell pipeline contain more than just the value of the item. They also include +properties such as the name and type of the item and of all the properties. Take, for example, the +.NET `DataTable` object. The header of a `DataTable` object contains the column (property) names and +types where each row in the `DataTable` only contains the value of each column. If you convert a +`DataTable` into a list of PowerShell objects, like: + +```PowerShell +$Data = $DataTable | Foreach-Object { $_ } +``` + +PowerShell converts each row into a new object, duplicating the header information for each row. The +memory usage considerably increases even if the value is just a few bytes. This extra overhead +shouldn't be an issue if you stream the objects through the pipeline because there will only be a +few objects in the pipeline at any time. + +### Missing properties + +Nevertheless, there is a pitfall in using the pipeline. Consider the following two objects being +output to a table: + +```PowerShell +$a = [pscustomobject]@{ name='John'; address='home'} +$b = [pscustomobject]@{ name='Jane'; phone='123'} +$a, $b |Format-Table +``` + +Results + +```Console +name address +---- ------- +John home +Jane +``` + +Notice that there is no `phone` column, meaning that the `phone='123'` property is missing from the +results. This is due to the one-at-a-time processing. At the moment the `Format-Table` cmdlet +receives object `$a` it is supposed to process it immediately by writing it to the console and +release it so that it can process the next item. The issue is that the `Format-Table` cmdlet is +unaware of the next object `$b` because it hasn't entered the pipeline yet. The initial output, +based on `$a`, has already been written to the console. In other words, a cmdlet written for +one-at-a-time processing bases its output on the first object received from the pipeline. This also +implies that if you change the order of the items in the pipeline (for example, +`$a, $b | Sort-Object | Format-Table`) properties might appear differently. + +### Processing blocks + +As you might have noticed, some actions, like outputting a header, are only required once. As in the +analogy with the assembly line, heating up a soldering gun is only required once, when the pipeline +is started. Cleaning up the station is only required when the pipeline is completed. Similar time +consuming or expensive actions could be required for a cmdlet, such as opening and closing a file. +These actions are respectively defined in the `Begin` and `End` blocks of a cmdlet. The actual +processing of items is defined in the `Process` block of cmdlet. A well defined pipeline PowerShell +cmdlet might look like this: + +```PowerShell +function MyCmdlet { + [CmdletBinding()] param( + [Parameter(ValueFromPipeLine = $True)] [String] $InputString + ) + Begin { + $Stream = [System.IO.StreamWriter]::new("$($Env:Temp)\My.Log") + } + Process { + $Stream.WriteLine($_) + } + End { + $Stream.Close() + } +} +``` + +When running this example cmdlet in a pipeline like `1..9 | MyCmdlet`, the log file is *only opened +once* at the start, then each item in the pipeline is processed one-at-a-time, and the log file is +closed (*once*) at the end. Note that when there are no `Begin`, `Process` and `End` processing +blocks defined in a function, the content of the function is assigned to the `End` block. See also: +[about Functions Advanced Methods][01]. + +A similar pipeline can be created with the common +[`Foreach-Object`][04] +cmdlet using the `-Begin`, `-Process` and `-End` parameters to define the corresponding process +blocks: + +```PowerShell +1..9 | Foreach-Object -Begin { + $Stream = [System.IO.StreamWriter]::new("$($Env:Temp)\My.Log") +} -Process { + $Stream.WriteLine($_) +} -End { + $Stream.Close() +} +``` + +### Performance + +With this understanding of the pipeline, you can see why you shouldn't wrap a cmdlet pipeline inside +another pipeline, like: + +```PowerShell +1..9 | ForEach-Object { + $_ | MyCmdlet +} +``` + +Wrapping a cmdlet pipeline into another (`ForEach-Object`) pipeline is very expensive because you're +also invoking the `begin` and `end` block of `MyCmdlet`. This will open and close the log file for +each item instead of only once at the beginning and the end of the pipeline. The performance +degradation can happend with any cmdlet that takes pipeline input. See also +[PowerShell scripting performance considerations][06]. + +## The steppable pipeline + +Unfortunately, it is not always possible to create a single syntactical pipeline. For example, you +might need different branches for different parameters values or as output paths. Consider a very +large `csv` file that you want to cut in smaller files. The obvious approach is to split it into +files with a maximum number of lines: + +```PowerShell +$BatchSize = 10000 +Import-Csv .\MyLarge.csv | + ForEach-Object -Begin { + $Index = 0 + } -Process { + $BatchNr = [math]::Floor($Index++/$BatchSize) + $_ | Export-Csv -Append .\Batch$BatchNr.csv + } +``` + +But as stated this before, this will open and close each output file (`.\Batch$BatchNr.csv` ) 10,000 +times where it only needs to be opened and closed once per output file. So, the solution here is to +use a steppable pipeline which lets you independently define the processing blocks for the required +output stream: + +```PowerShell +$BatchSize = 10000 +Import-Csv .\MyLarge.csv | + ForEach-Object -Begin { + $Index = 0 + } -Process { + if ($Index % $BatchSize -eq 0) { + $BatchNr = [math]::Floor($Index++/$BatchSize) + $Pipeline = { Export-Csv -notype -Path .\Batch$BatchNr.csv }.GetSteppablePipeline() + $Pipeline.Begin($True) + } + $Pipeline.Process($_) + if ($Index++ % $BatchSize -eq 0) { $Pipeline.End() } + } -End { + $Pipeline.End() + } +``` + +Every 10,000 (`$BatchSize`) entries, the modulus (`%`) is zero and a new pipeline is created for the +expression `{ Export-Csv -notype -Path .\Batch$BatchNr.csv }`. + +* The `$Pipeline.Begin($True)` invokes the `Begin` block of the steppable pipeline, which opens an + new `csv` file named `.\Batch$BatchNr.csv` and writes the headers to the file. +* The `$Pipeline.Process($_)` invokes the `Process` block of the steppable pipeline using the + current item (`$_`), which is appended to the `csv` file. +* The `$Pipeline.End()`invokes the `End` block of the steppable pipeline, which closes the `csv` + file named `.\Batch$BatchNr.csv`. This file holds a total of 10,000 entries. + +(Note that it is important to end the pipeline but there is no harm in invoking the +`$Pipeline.End()` multiple times.) + +It is a little more code, but if you measure the results you will see that in this situation the +later script is more than 50 times faster than the one with the wrapped cmdlet pipeline. + +### Multiple output pipelines + +With the steppable pipeline technique, you might even have multiple output pipelines open at once. +Consider that for the very large `csv` file in the previous example, you do not want batches of +10,000 entries but divide the entries over 26 files based on the first letter of the `LastName` +property: + +```PowerShell +$Pipeline = @{} +Import-Csv .\MyLarge.csv | + ForEach-Object -Process { + $Letter = $_.LastName[0].ToString().ToUpper() + if (!$Pipeline.Contains($Letter)) { + $Pipeline[$Letter] = { Export-CSV -notype -Path .\$Letter.csv }.GetSteppablePipeline() + $Pipeline[$Letter].Begin($True) + } + $Pipeline[$Letter].Process($_) + } -End { + foreach ($Key in $Pipeline.Keys) { $Pipeline[$Key].End() } + } +``` + +**Explanation:** + +* `Import-Csv .\MyLarge.csv | ForEach-Object -Process {` + + processes each (One-at-a-time) item of the `csv` file + +* `$Letter = $_.LastName[0].ToString().ToUpper()` + + Takes the first character of the `LastName` property and puts that in upper case. + +* `if (!$Pipeline.Contains($Letter)) {` + + If the pipeline for the specific character doesn't yet exist: + + * Open a new steppable pipeline for the specific letter: + `{ Export-CSV -notype -Path .\$Letter.csv }.GetSteppablePipeline()` + * And invoke the `Begin` block: `.Begin($True)` which creates a new `csv` file with the concerned + headers + +* `foreach ($Key in $Pipeline.Keys) { $Pipeline[$Key].End() }` + + Closes all the existing steppable pipelines (aka `csv` files) + +### See also + +* [About pipelines][02] +* [Cmdlet Overview][07] +* [Strongly Encouraged Development Guidelines][08] +* [about Functions Advanced Methods][01] +* [PowerShell scripting performance considerations][05] + +<!-- link references --> +[01]: https://learn.microsoft.com/powershell/module/microsoft.powershell.core/about/about_functions_advanced_methods +[02]: https://learn.microsoft.com/powershell/module/microsoft.powershell.core/about/about_pipelines +[03]: https://learn.microsoft.com/powershell/module/microsoft.powershell.core/about/about_pipelines#one-at-a-time-processing +[04]: https://learn.microsoft.com/powershell/module/microsoft.powershell.core/foreach-object +[05]: https://learn.microsoft.com/powershell/scripting/dev-cross-plat/performance/script-authoring-considerations +[06]: https://learn.microsoft.com/powershell/scripting/dev-cross-plat/performance/script-authoring-considerations#avoid-wrapping-cmdlet-pipelines +[07]: https://learn.microsoft.com/powershell/scripting/developer/cmdlet/cmdlet-overview +[08]: https://learn.microsoft.com/powershell/scripting/developer/cmdlet/strongly-encouraged-development-guidelines diff --git a/Posts/2023/03/Update-XML-File-using-PowerShell.md b/Posts/2023/03/Update-XML-File-using-PowerShell.md new file mode 100644 index 00000000..2d699e6c --- /dev/null +++ b/Posts/2023/03/Update-XML-File-using-PowerShell.md @@ -0,0 +1,136 @@ +--- +post_title: Update XML files using PowerShell +username: sorastog +categories: PowerShell +tags: PowerShell, XML, Configuration +summary: This posts explains how to update XML files using PowerShell +--- + +There are many blogs on internet already speaking about updating XML files in PowerShell, but I +felt need of one consolidated blog where complex XML files can also be updated with long complex +hierarchy of XML nodes and attributes. + +Below is an XML example which we will try in this blog to update at various level of node hierarchy. + +## Sample Code + +```xml +<?xml version="1.0" encoding="utf-8"?> +<Data version="2.0"> + <Roles> + <Role Name="ManagementServer" Value="OldManagementServer" /> + </Roles> + <SQL> + <Instance Server="OldSQLServer" Instance="MSSQLSERVER" Version="SQL Server 2012"> + <Variable Name="SQLAdmin" Value="Domain\OldSQlAdmin" /> + <Variable Name="SQLUser" Value="domain\sqluser" /> + </Instance> + </SQL> + <VMs> + <VM Type="ClientVM"> + <VMName>ClientVM</VMName> + </VM> + <VM Type="DNSServerVM"> + <VMName>OldDNSServer</VMName> + </VM> + </VMs> +</Data> +``` + +## Steps to follow + +We will update the nodes in this XML file to use a new management, SQL, and DNS servers. Below are +the steps given separately on how we can update the nodes and their attributes at various levels. + +1. Define the variables which need to be modified: + + ```powershell + $path = 'C:\Users\sorastog\Desktop\blog\Variable.xml' + $ManagementServer = 'NewManagementServer' + $SQLServer = 'NewSQLServer' + $SQLAdmin = 'Domain\NewSQlAdmin' + $DNSServerVMName = 'NewDNSServer' + ``` + +1. Reading the content of XML file. + + ```powershell + $xml = [xml](Get-Content -Path $path) + ``` + +1. Update `ManagementServer`: Change the attribute **Value** of nodes at level 3 based on the + **Name** attribute on the same level. + + ```powershell + $node = $xml.Data.Roles.Role | + Where-Object -Process { $_.Name -eq 'ManagementServer' } + $node.Value = $ManagementServer + ``` + +1. Update `SQLServer`: Change the attribute **Value** of a node at level 3. + + ```powershell + $node = $xml.Data.SQL.Instance + $node.Server = $SQLServer + ``` + +1. Update `SQLAdmin`: Change the attribute **Value** of nodes at level 4 based on the **Name** + attribute on the same level. + + ```powershell + $node = $xml.Data.SQL.Instance.Variable | + Where-Object -Process { $_.Name -eq 'SQLAdmin' } + $node.Value = $SQLAdmin + ``` + +1. Update `DNSServerVM`: Change the attribute **Value** of nodes at level 4 based on the **VMType** + attribute at the level above. + + ```powershell + $node = $xml.Data.VMs.VM | + Where-Object -Process { $_.Type -eq 'DNSServerVM' } + $node.VMName = $DNSServerVMName + ``` + +1. Save changes to the XML file. + + ```powershell + $xml.Save($path) + ``` + +## Output + +The final PowerShell script would look like below: + +```powershell +$path = 'C:\Data.xml' +$ManagementServer = 'NewManagementServer' +$SQLServer = 'NewSQLServer' +$SQLAdmin = 'Domain\NewSQlAdmin' +$DNSServerVMName = 'NewDNSServer' + +$xml = [xml](Get-Content $path) + +$node = $xml.Data.Roles.Role | + Where-Object -Process { $_.Name -eq 'ManagementServer' } +$node.Value = $ManagementServer + +$node = $xml.Data.SQL.Instance +$node.Server = $SQLServer + +$node = $xml.Data.SQL.Instance.Variable | + Where-Object -Process { $_.Name -eq 'SQLAdmin' } +$node.Value = $SQLAdmin + +$node = $xml.Data.VMs.VM | + Where-Object -Process { $_.Type -eq 'DNSServerVM' } +$node.VMName = $DNSServerVMName + +$xml.Save($path) +``` + +Hope this will help you to update even complex XML files with multiple nodes and deep hierarchies. +If there are some XML nodes that you would like to update and the category is not included in this +blog, please reply to this post and I will add it. + +Till Then, Happy Scripting :) diff --git a/Posts/2023/04/Convert-Specific-Sheet-Of-Excel-Into-Json-Using-PowerShell.md b/Posts/2023/04/Convert-Specific-Sheet-Of-Excel-Into-Json-Using-PowerShell.md new file mode 100644 index 00000000..8b0086f5 --- /dev/null +++ b/Posts/2023/04/Convert-Specific-Sheet-Of-Excel-Into-Json-Using-PowerShell.md @@ -0,0 +1,369 @@ +--- +post_title: Convert Specific Table of Excel Sheet to JSON +username: sorastog +categories: PowerShell +tags: PowerShell, Excel, Json, Automation +summary: This posts explains how to Convert Specific Table of Excel Sheet to JSON +--- + +There is an excellent [script on GitHub][01] that helps to convert a full Excel sheet to JSON. The +script expects the table to be at the start of the sheet; that is, to have the first header in the +`A1` cell. + +I had a little different requirement. I had to convert a specific table among various tables +available within a sheet in an Excel file as shown in image below. + +![Screenshot of an Excel sheet showing a table in the middle of a sheet instead of at the start][02] + +Our requirement is to read `Class 6` students' data. In the above screenshot, there are multiple +sheets within the Excel workbook. There are multiple tables like `Class 1`, `Class 2`, and so +on inside the **Science** sheet. + +As our requirement is to read `Class 6` students data from **Science** sheet, lets look closely at +how the data is available in the Excel sheet. + +- The name of the class is at row 44. +- The column headers are on row 45. +- The data starts from row 46. + +[alert type="note" heading="Note"] +The tables can be at any location (any column and any row) within the sheet. The only fixed +identifier is **ClassName** which is `Class 6` in this example. +[/alert] + +## Steps to follow + +Follow these steps to see how you can read `Class 6` data from **Science** sheet: + +1. Handle input parameters. + + The script accepts 3 parameters: + + - `$InputFileFullPath` - This is path of the input Excel file. + - `$SubjectName` - This is name of the sheet inside the Excel file. + - `$ClassName` - This is name of the table within the Excel sheet. + + ```powershell + $InputFileFullPath = 'C:\Data\ABCDSchool.xlsx' + $SubjectName = 'Science' + $ClassName = 'Class 6' + ``` + +1. Open the Excel file and read the **Science** sheet. + + ```powershell + $excelApplication = New-Object -ComObject Excel.Application + $excelApplication.DisplayAlerts = $false + $Workbook = $excelApplication.Workbooks.Open($InputFileFullPath) + + $sheet = $Workbook.Sheets | Where-Object { $_.Name -eq $SubjectName } + + if (-not $sheet) { + throw "Could not find subject '$SubjectName' in the workbook" + } + ``` + +1. Grab the `Class 6` table within the **Science** sheet to work with. + + ```powershell + # Find the cell where Class name is mentioned + $found = $sheet.Cells.Find($ClassName) + $beginAddress = $Found.Address(0, 0, 1, 1).Split('!')[1] + $beginRowAddress = $beginAddress.Substring(1, 2) + # Header row starts 1 row after the class name + $startHeaderRowNumber = [int]$beginRowAddress + 1 + # Student data row starts 1 row after header row + $startDataRowNumber = $startHeaderRowNumber + 1 + $beginColumnAddress = $beginAddress.Substring(0, 1) + # ASCII number of column + $startColumnHeaderNumber = [BYTE][CHAR]$beginColumnAddress - 65 + 1 + ``` + +1. Extract the header column names (**Logical Seat Location**, **Actual Seat Location**, + **LAN Port #**, **Monitor Cable Port**, **Student Name**, **Student#**, and **Room Type**) + + ```powershell + $Headers = @{} + $numberOfColumns = 0 + $foundHeaderValue = $true + + while ($foundHeaderValue -eq $true) { + $headerCellValue = $sheet.Cells.Item( + $startHeaderRowNumber, + ($numberOfColumns + $startColumnHeaderNumber) + ).Text + + if ($headerCellValue.Trim().Length -eq 0) { + $foundHeaderValue = $false + } else { + $numberOfColumns++ + if ($Headers.ContainsValue($headerCellValue)) { + # Do not add any duplicate column again. + } else { + $Headers.$numberOfColumns = $headerCellValue + } + } + } + ``` + +1. Extract the data (`Class 6` student information rows). + + ```powershell + $results = @{} + $rowNumber = $startDataRowNumber + $finish = $false + + while ($finish -eq $false) { + if ($rowNumber -gt 1) { + $result = @{} + + foreach ($columnNumber in $Headers.GetEnumerator()) { + $columnName = $columnNumber.Value + # Student data row, student data column number + $cellValue = $sheet.Cells.Item( + $rowNumber, + ($columnNumber.Name + ($startColumnHeaderNumber -1 )) + ).Value2 + + if ($cellValue -eq $null) { + $finish = $true + break; + } + + $result.Add($columnName.Trim(), $cellValue.Trim()) + } + + if ($finish -eq $false) { + # Adding Excel sheet row number for validation + $result.Add("RowNumber",$rowNumber) + $results += $result + $rowNumber++ + } + } + } + ``` + +1. Create the JSON file and close the Excel file. + + ```powershell + $inputFileName = Split-Path $InputFileFullPath -leaf + $inputFileName = $inputFileName.Split('.')[0] + # Output file name will be "ABCDSchool-Science-Class 6.json" + $jsonOutputFileName = "$inputFileName-$SubjectName-$ClassName.json" + $jsonOutputFileFullPath = [System.IO.Path]::GetFullPath($jsonOutputFileName) + + Write-Host "Converting sheet '$SubjectName' to '$jsonOutputFileFullPath'" + + $null = $results | + ConvertTo-Json | + Out-File -Encoding ASCII -FilePath $jsonOutputFileFullPath + $null = $excelApplication.Workbooks.Close() + $null = [System.Runtime.InteropServices.Marshal]::ReleaseComObject( + $excelApplication + ) + ``` + +## Putting it all together + +The full code goes like this: + +```powershell +param ( + # Excel name + [Parameter(Mandatory=$true)] + [string]$InputFileFullPath, + # Sheet name + [Parameter(Mandatory=$true)] + [string]$SubjectName, + # Identifier for the table + [Parameter(Mandatory=$true)] + [string]$ClassName +) + +#region Open Excel file +$excelApplication = New-Object -ComObject Excel.Application +$excelApplication.DisplayAlerts = $false +$Workbook = $excelApplication.Workbooks.Open($InputFileFullPath) + +# Find sheet +$sheet = $Workbook.Sheets | Where-Object { $_.Name -eq $SubjectName } + +if (-not $sheet) { + throw "Could not find subject '$SubjectName' in the workbook" +} +#endregion Open Excel file + +#region Grab the table within sheet to work with +# Find the cell where Class name is mentioned +$found = $sheet.Cells.Find($ClassName) +$beginAddress = $Found.Address(0, 0, 1, 1).Split('!')[1] +$beginRowAddress = $beginAddress.Substring(1, 2) +# Header row starts 1 row after the class name +$startHeaderRowNumber = [int]$beginRowAddress + 2 +# Student data row starts 1 row after header row +$startDataRowNumber = $startHeaderRowNumber + 1 +$beginColumnAddress = $beginAddress.Substring(0,1) +# ASCII number of column +$startColumnHeaderNumber = [BYTE][CHAR]$beginColumnAddress - 65 + 1 +#endregion Grab the table within sheet to work with + +#region Extract Header Columns Name +$Headers = @{} +$numberOfColumns = 0 +$foundHeaderValue = $true + +while ($foundHeaderValue -eq $true) { + $headerCellValue = $sheet.Cells.Item( + $startHeaderRowNumber, + ($numberOfColumns + $startColumnHeaderNumber) + ).Text + + if ($headerCellValue.Trim().Length -eq 0) { + $foundHeaderValue = $false + } else { + $numberOfColumns++ + if ($Headers.ContainsValue($headerCellValue)) { + # Do not add any duplicate column again. + } else { + $Headers.$numberOfColumns = $headerCellValue + } + } +} +#endregion Extract Header Columns Name + +#region Extract Student Information Rows +$results = @() +$rowNumber = $startDataRowNumber +$finish = $false + +while ($finish -eq $false) { + if ($rowNumber -gt 1) { + $result = @{} + + foreach ($columnNumber in $Headers.GetEnumerator()) { + $columnName = $columnNumber.Value + # Student data row, student data column number + $cellValue = $sheet.Cells.Item( + $rowNumber, + ($columnNumber.Name + ($startColumnHeaderNumber - 1)) + ).Value2 + + if ($cellValue -eq $null) { + $finish = $true + break + } + + $result.Add($columnName.Trim(),$cellValue.Trim()) + } + + if ($finish -eq $false) { + # Adding Excel sheet row number for validation + $result.Add("RowNumber",$rowNumber) + $results += $result + $rowNumber++ + } + } +} +#endregion Extract Student Information Rows + +#region Create JSON file and close Excel file +$inputFileName = Split-Path $InputFileFullPath -leaf +$inputFileName = $inputFileName.Split('.')[0] +# Output file name will be "ABCDSchool-Science-Class 6.json" +$jsonOutputFileName = "$inputFileName-$SubjectName-$ClassName.json" +$jsonOutputFileFullPath = [System.IO.Path]::GetFullPath($jsonOutputFileName) + +Write-Host "Converting sheet '$SubjectName' to '$jsonOutputFileFullPath'" + +$null = $results | + ConvertTo-Json | + Out-File -Encoding ASCII -FilePath $jsonOutputFileFullPath +$null = $excelApplication.Workbooks.Close() +$null = [System.Runtime.InteropServices.Marshal]::ReleaseComObject( + $excelApplication +) +#endregion Create JSON file and close Excel file +``` + +The output JSON file will look like below: + +```json +[ + { + "Room Type": "Standard", + "RowNumber": 46, + "Student Name": "Alex", + "Student#": "RL45", + "LAN Port #": "LAN Port 7-8", + "Logical Seat Location": "SL 11", + "Actual Seat Location": "Seat43", + "Monitor Cable Port": "C-D" + }, + { + "Room Type": "Standard", + "RowNumber": 47, + "Student Name": "Alex", + "Student#": "RL45", + "LAN Port #": "LAN Port 5-6", + "Logical Seat Location": "SL 11", + "Actual Seat Location": "Seat43", + "Monitor Cable Port": "A-B" + }, + { + "Room Type": "Standard", + "RowNumber": 48, + "Student Name": "John", + "Student#": "RL47", + "LAN Port #": "LAN Port 3-4", + "Logical Seat Location": "SL 11", + "Actual Seat Location": "Seat43", + "Monitor Cable Port": "C-D" + }, + { + "Room Type": "Standard", + "RowNumber": 49, + "Student Name": "John", + "Student#": "RL47", + "LAN Port #": "LAN Port 1-2", + "Logical Seat Location": "SL 11", + "Actual Seat Location": "Seat43", + "Monitor Cable Port": "A-B" + }, + { + "Room Type": "Standard", + "RowNumber": 50, + "Student Name": "Victor", + "Student#": "RL35", + "LAN Port #": "LAN Port 7-8", + "Logical Seat Location": "SL 10", + "Actual Seat Location": "Seat33", + "Monitor Cable Port": "C-D" + }, + { + "Room Type": "Standard", + "RowNumber": 51, + "Student Name": "Victor", + "Student#": "RL35", + "LAN Port #": "LAN Port 5-6", + "Logical Seat Location": "SL 10", + "Actual Seat Location": "Seat33", + "Monitor Cable Port": "A-B" + }, + { + "Room Type": "Standard", + "RowNumber": 52, + "Student Name": "Honey", + "Student#": "RL42", + "LAN Port #": "LAN Port 3-4", + "Logical Seat Location": "SL 10", + "Actual Seat Location": "Seat33", + "Monitor Cable Port": "C-D" + } +] +``` + +Feel free to drop your feedback and inputs on this page. Till then, Happy Scripting!!! + +<!-- Link Reference Definitions --> +[01]: https://github.com/chrisbrownie/Convert-ExcelSheetToJson/blob/master/Convert-ExcelSheetToJson.ps1 +[02]: ./media/Convert-Specific-Sheet-Of-Excel-Into-Json-Using-PowerShell/Image-MultipleTablesInOneSheet.png diff --git a/Posts/2023/04/media/Convert-Specific-Sheet-Of-Excel-Into-Json-Using-PowerShell/Image-MultipleTablesInOneSheet.png b/Posts/2023/04/media/Convert-Specific-Sheet-Of-Excel-Into-Json-Using-PowerShell/Image-MultipleTablesInOneSheet.png new file mode 100644 index 00000000..1f04c700 Binary files /dev/null and b/Posts/2023/04/media/Convert-Specific-Sheet-Of-Excel-Into-Json-Using-PowerShell/Image-MultipleTablesInOneSheet.png differ diff --git a/Posts/2023/05/Designing-For-User-Experience-In-PowerShell.md b/Posts/2023/05/Designing-For-User-Experience-In-PowerShell.md new file mode 100644 index 00000000..d6a722c3 --- /dev/null +++ b/Posts/2023/05/Designing-For-User-Experience-In-PowerShell.md @@ -0,0 +1,106 @@ +--- +post_title: Designing PowerShell For End Users +username: svalding +categories: PowerShell +tags: PowerShell, Design, Toolmaking, User Experience +summary: This posts explains taking user experience into account when designing PowerShell tools +--- + +PowerShell, being built on .NET and object-oriented in nature, is a _fantastic_ language for developing +tooling that you can deliver to your end users. These may be fellow technologists, or they could also be +non-technical users within your organization. This could also be a tool you wish to share with the community, +either via your own Github or by publishing to the PowerShell Gallery. + +## What Are You Doing? + +When setting out with the task of developing a tool you should, as a first step, stop and think. Think about +what problem your tool is trying to solve. This could be a number of things + +- Creating data +- collating data +- Interacting with a system or systems + +The sky is the limit here, but your first thing is to determine what it +is that you are trying to accomplish. + +## What Should You Call It? + +Your second step should be to consider your tool's name. Whether this is a single function, or a series of functions +that form a new module, you should consider the following: + +- Use [approved verbs](https://learn.microsoft.com/powershell/scripting/developer/cmdlet/approved-verbs-for-windows-powershell-commands) for Functions. You can run `Get-Verb` in your console to quickly get a list! _Tip_: Use `Get-Verb | Sort-Object` to make this easier to parse! +- Use a coherent noun. Be as specific as possible. Using a great combination of verb/noun syntax provides clarity + to what your tool does. + +## Designing Parameters + +This step _could_ take some time, and a little trial and error. You want your tool to be flexible, but you don't want your parameter +names to be so difficult such that they are hard to use/remember. Succinct is better here. If you need to add some flexibility to your +tool, considering using [ParameterSets](https://learn.microsoft.com/powershell/module/microsoft.powershell.core/about/about_parameter_sets). These will give your end users a few different ways to use your tool, if that is or becomes necessary in the future. + +### Applying Guardrails + +Guardrails, in this context, refers to the application of restrictions upon your parameters. These prevent your end users from passing incorrect input +to the tool you've provided them. Given that PowerShell is built on .NET, there is a _ton_ of flexibility and strength in the guardrails you can employ. + +I'll touch on just a few of my favorites, but this is by far not an exhaustive list. + +#### 1. ValidateSet + +Let's look at an example first: + +```powershell +[CmdletBinding()] +Param( + [Parameter()] + [ValidateSet('Cat','Dog','Fish','Bird')] + [String] + $Animal +) +``` + +If you notice above, we've defined a non-mandatory parameter that is of type `[String]`. If you notice above, we've defined a non-mandatory parameter that is of type `[String]`. This is a guardrail because any other type causes an error to be thrown. +We have added further restrictions (guardrails) on this parameter by employing a `[ValidateSet()]` attribute, which limits the valid input to _only_ those items that are +a member of the set. Provide `Horse` to the animal parameter and, even though it is a string, it produces an error because it's not a member of the approved set of inputs. + +#### 2. ValidateRange + +We'll start with another example: + +```powershell +[CmdletBinding()] +Param( + [Parameter()] + [ValidateRange(2005,2023)] + [Int] + $Year +) +``` + +In this example we have defined a `Year` parameter that is an `[Int]`, meaning only numbers are valid input. We've applied guardrails via `[ValidateRange()]`, which limits the input to between 2005 and 2023. Any number outside of that range produces an error. + +#### 3. ValidateScript + +The `[ValidateScript()]` attribute is extremely powerful. It allows you to run arbitrary PowerShell code in a script block to check the input of a given parameter. +Let's check out a _very_ simple example: + +```powershell +[CmdletBinding()] +Param( + [Parameter()] + [ValidateScript({ Test-Path $_ })] + [String] + $InputFile +) +``` + +By using `Test-Path $_` in the Scriptblock of our `[ValidateScript()]` attribute we are instructing +PowerShell to confirm that the input we have provided to the parameter actually exists (_Notice the +addition of `{}` here_). This helps by putting guardrails around human error in the form of typos. + +## Wrapping It Up + +As previously stated, adding guardrails to your tools using these methods (and countless others not mentioned) _demonstrably_ increases the usability and adoption of your tools. + +So take a step back, think about your tool's design _first_, and then start writing the code. I +think you'll find that it is a much more enjoyable experience, from creation to adoption. diff --git a/Posts/2023/05/Measuring-Script-Execution-Time.md b/Posts/2023/05/Measuring-Script-Execution-Time.md new file mode 100644 index 00000000..29270ac2 --- /dev/null +++ b/Posts/2023/05/Measuring-Script-Execution-Time.md @@ -0,0 +1,280 @@ +--- +post_title: 'Measuring script execution time' +username: francisconabas +categories: PowerShell +tags: PowerShell, Automation, Performance, Measure-Command +summary: This post shows how to measure script execution time in PowerShell +--- + +Most of the time while developing PowerShell scripts we don't need to worry about performance, or +execution time. After all, scripts were made to run automation in the background. However, as your +scripts become more sophisticated, and you need to work with complex data or big data sizes, +performance becomes something to keep in mind. Measuring a script execution time is the first step +towards script optimization. + +## Measure-Command + +PowerShell has a built-in cmdlet called `Measure-Command`, which measures the execution time of +other cmdlets, or script blocks. It has two parameters: + +- **Expression**: The script block to be measured. +- **InputObject**: Optional input to be passed to the script block. You can use `$_` or `$PSItem` to + access them. + +Besides the two parameters, objects in the pipeline are also passed to the script block. +`Measure-Command` returns an object of type `System.TimeSpan`, giving us more flexibility on how to +work with the result. + +```powershell +Measure-Command { foreach ($number in 1..1000) { <# Do work #> } } +``` + +```powershell-console +Days : 0 +Hours : 0 +Minutes : 0 +Seconds : 0 +Milliseconds : 8 +Ticks : 85034 +TotalDays : 9.84189814814815E-08 +TotalHours : 2.36205555555556E-06 +TotalMinutes : 0.000141723333333333 +TotalSeconds : 0.0085034 +TotalMilliseconds : 8.5034 +``` + +Using the pipeline or the **InputObject** parameter. + +```powershell +1..1000 | + Measure-Command -Expression { foreach ($number in $_) { <# Do work #> } } | + Select-Object TotalMilliseconds +``` + +```powershell-console +TotalMilliseconds +----------------- + 10.60 +``` + +```powershell +Measure-Command -InputObject (1..1000) -Expression { $_ | % { <# Do work #> } } | + Select-Object TotalMilliseconds +``` + +```powershell-console +TotalMilliseconds +----------------- + 19.98 +``` + +## Scope and Object Modification + +`Measure-Command` runs the script block in the current scope, meaning variables in the current scope +gets modified if referenced in the script block. + +```powershell +$studyVariable = 0 +Measure-Command { 1..10 | % { $studyVariable += 1 } } +Write-Host "Current variable value: $studyVariable." +``` + +```powershell-console +Days : 0 +Hours : 0 +Minutes : 0 +Seconds : 0 +Milliseconds : 15 +Ticks : 155838 +TotalDays : 1.80368055555556E-07 +TotalHours : 4.32883333333333E-06 +TotalMinutes : 0.00025973 +TotalSeconds : 0.0155838 +TotalMilliseconds : 15.5838 + +Current variable value: 10. +``` + +To overcome this, you can use the invocation operator `&` and enclose the script block in `{}`, to +execute in a separate context. + +```powershell +$studyVariable = 0 +Measure-Command { & { 1..10 | % { $studyVariable += 1 } } } +Write-Host "Current variable value: $studyVariable." +``` + +```powershell-console +Days : 0 +Hours : 0 +Minutes : 0 +Seconds : 0 +Milliseconds : 8 +Ticks : 86542 +TotalDays : 1.00164351851852E-07 +TotalHours : 2.40394444444444E-06 +TotalMinutes : 0.000144236666666667 +TotalSeconds : 0.0086542 +TotalMilliseconds : 8.6542 + +Current variable value: 0. +``` + +It's also worth remember that if your script block modifies system resources, files, databases or +any other static data, the object gets modified. + +```powershell +$scriptBlock = { + if (!(Test-Path -Path C:\SuperCoolFolder)) { + New-Item -Path C:\ -Name SuperCoolFolder -ItemType Directory + } +} + +Measure-Command -Expression { & $scriptBlock } +Get-ChildItem C:\ -Filter SuperCoolFolder | Select-Object FullName +``` + +```powershell-console +Days : 0 +Hours : 0 +Minutes : 0 +Seconds : 0 +Milliseconds : 11 +Ticks : 118978 +TotalDays : 1.37706018518519E-07 +TotalHours : 3.30494444444444E-06 +TotalMinutes : 0.000198296666666667 +TotalSeconds : 0.0118978 +TotalMilliseconds : 11.8978 + +FullName : C:\SuperCoolFolder +``` + +As a cool exercise, try figuring out why the output from `New-Item` didn't show up. + +## Output and Alternatives + +`Measure-Command` returns a `System.TimeSpan` object, but not the result from the script. If your +study also includes the result, there are two ways you can go about it. + +### Saving the output in a variable + +We know that scripts executed with `Measure-Object` runs in the current scope. So we could assign +the result to a variable, and work with it. + +```powershell +$range = 1..100 +$evenCount = 0 +$scriptBlock = { + foreach ($number in $range) { + if ($number % 2 -eq 0) { + $evenCount++ + } + } +} + +Measure-Command -InputObject (1..100) -Expression $scriptBlock | + Format-List TotalMilliseconds +Write-Host "The count of even numbers in 1..100 is $evenCount." +``` + +```powershell-console +TotalMilliseconds : 1.3838 + +The count of even numbers in 1..100 is 50. +``` + +### Custom Function + +If you are serious about the performance variable, and want to keep the script block as clean as +possible, we could elaborate our own function, and shape the output as we want. + +The `Measure-Command` Cmdlet uses an object called `System.Diagnostics.Stopwatch`. It works like a +real stopwatch, and we control it using its methods, like `Start()`, `Stop()`, etc. All we need to +do is start it before executing our script block, stop it after execution finishes, and collect the +result from the **Elapsed** property. + +```powershell +function Measure-CommandEx { + + [CmdletBinding()] + param ( + [Parameter(Mandatory, Position = 0)] + [scriptblock]$Expression, + + [Parameter(ValueFromPipeline)] + [psobject[]]$InputObject + ) + + Begin { + $stopWatch = New-Object -TypeName 'System.Diagnostics.Stopwatch' + + <# + We need to define result as a list because the way objects + are passed to the pipeline. If you pass a collection of objects, + the pipeline sends them one by one, and the result + is always overridden by the last item. + #> + [System.Collections.Generic.List[PSObject]]$result = @() + } + + Process { + if ($InputObject) { + + # Starting the stopwatch. + $stopWatch.Start() + + # Creating the '$_' variable. + $dollarUn = New-Object -TypeName psvariable -ArgumentList @('_', $InputObject) + + <# + Overload is: + InvokeWithContext( + Dictionary<string, scriptblock> functionsToDefine, + List<psvariable> variablesToDefine, + object[] args + ) + #> + $result.AddRange($Expression.InvokeWithContext($null, $dollarUn, $null)) + + $stopWatch.Stop() + } + else { + $stopWatch.Start() + $result.AddRange($Expression.InvokeReturnAsIs()) + $stopWatch.Stop() + } + } + + End { + return [PSCustomObject]@{ + ElapsedTimespan = $stopWatch.Elapsed + Result = $result + } + } +} +``` + +Note that there is overhead when using the **InputObject** parameter, meaning there is a +difference in the overall execution time. + +## Conclusion + +I hope you, like me, learned something new today, and had fun along the way. + +Until a next time, happy scripting! + +## Links + +- [Measure-Command][01] +- [InvokeWithContext Method][02] +- [InvokeReturnAsIs Method][03] +- [Test our WindowsUtils module!][04] +- [See what I'm up to][05] + +<!-- link references --> +[01]: https://learn.microsoft.com/powershell/module/microsoft.powershell.utility/measure-command +[02]: https://learn.microsoft.com/dotnet/api/system.management.automation.scriptblock.invokewithcontext +[03]: https://learn.microsoft.com/dotnet/api/system.management.automation.scriptblock.invokereturnasis +[04]: https://github.com/FranciscoNabas/WindowsUtils +[05]: https://github.com/FranciscoNabas diff --git a/Posts/2023/05/Media/Porting-GeneratePassword-From-Csharp/File-OpenFromGAC.png b/Posts/2023/05/Media/Porting-GeneratePassword-From-Csharp/File-OpenFromGAC.png new file mode 100644 index 00000000..fe6f8492 Binary files /dev/null and b/Posts/2023/05/Media/Porting-GeneratePassword-From-Csharp/File-OpenFromGAC.png differ diff --git a/Posts/2023/05/Media/Porting-GeneratePassword-From-Csharp/GeneratePasswordMethod.png b/Posts/2023/05/Media/Porting-GeneratePassword-From-Csharp/GeneratePasswordMethod.png new file mode 100644 index 00000000..f68a0dd9 Binary files /dev/null and b/Posts/2023/05/Media/Porting-GeneratePassword-From-Csharp/GeneratePasswordMethod.png differ diff --git a/Posts/2023/05/Media/Porting-GeneratePassword-From-Csharp/MembershipClass.png b/Posts/2023/05/Media/Porting-GeneratePassword-From-Csharp/MembershipClass.png new file mode 100644 index 00000000..bf48203d Binary files /dev/null and b/Posts/2023/05/Media/Porting-GeneratePassword-From-Csharp/MembershipClass.png differ diff --git a/Posts/2023/05/Media/Porting-GeneratePassword-From-Csharp/OpenFromGACMenu.png b/Posts/2023/05/Media/Porting-GeneratePassword-From-Csharp/OpenFromGACMenu.png new file mode 100644 index 00000000..d5fc345f Binary files /dev/null and b/Posts/2023/05/Media/Porting-GeneratePassword-From-Csharp/OpenFromGACMenu.png differ diff --git a/Posts/2023/05/Media/Porting-GeneratePassword-From-Csharp/Result.png b/Posts/2023/05/Media/Porting-GeneratePassword-From-Csharp/Result.png new file mode 100644 index 00000000..03e7ed43 Binary files /dev/null and b/Posts/2023/05/Media/Porting-GeneratePassword-From-Csharp/Result.png differ diff --git a/Posts/2023/05/Porting-GeneratePassword-From-Csharp.md b/Posts/2023/05/Porting-GeneratePassword-From-Csharp.md new file mode 100644 index 00000000..42111ee6 --- /dev/null +++ b/Posts/2023/05/Porting-GeneratePassword-From-Csharp.md @@ -0,0 +1,607 @@ +--- +post_title: 'Porting System.Web.Security.Membership.GeneratePassword() to PowerShell' +username: francisconabas +categories: PowerShell +post_slug: porting-system-web-security-membership-generatepassword-to-powershell +tags: PowerShell, Automation, Password, Portability, C# +summary: This post shows how to port a C# method into PowerShell +--- +<!-- markdownlint-disable-file MD041 --> +I've been using PowerShell (core) for a couple of years now, and it became natural to create +automations with all the features that are not present in Windows PowerShell. However, there is +still one feature I miss in PowerShell, and this feature, for as silly as it sounds, is the +**GeneratePassword**, from **System.Web.Security.Membership**. + +This happens because this assembly was developed in .NET Framework, and not brought to .NET (core). +Although there are multiple alternatives to achieve the same result, I thought this is the perfect +opportunity to show the Power in PowerShell, and port this method from C#. + +## Method + +We are going to get this method's code by using an IL decompiler. C# is compiled to an +**Intermediate Language**, which allows us to decompile it. The tool I'll be using is `ILSpy`, and +can be found on the [Microsoft Store][09]. + +[alert type="note" title="Disclaimer"] The code for **GeneratePassword** and the **System.Web** +library were not written by me, and the purpose of decompiling it is purely educational. For as +harmless as this code is, it does not have any security warranties, nor is intended for misuse. +[/alert] + +## Getting the Code + +Once installed, open `ILSpy`, click on **File** and **Open from GAC...**. On the search bar, type +**System.Web**, select the assembly, and click **Open**. + +![File menu][01] +![Open from GAC menu][04] + +Once loaded, expand the **System.Web** assembly tree, and the **System.Web.Security** namespace. +Inside **System.Web.Security**, look for the **Membership** class, click on it, and the decompiled +code should appear on the right pane. + +![Membership class][03] + +Scroll down until you find the **GeneratePassword** method, and expand it. + +![GeneratePassword method][02] + + +## Porting to PowerShell + +Now the fun begins. Let's do this using PowerShell tools only, means we're not going to copy the +**Membership** class and method. We are going to create a function, and keep the variable names the +same, so it's easier for us to compare. + +- Starting with the method's signature: + `public static string GeneratePassword(int lenght, int numberOfNonAlphanumericCharacters)` + - **public** means this method can be called from outside the assembly. + - **static** means I can call this method without having to instantiate an object of type + **Membership**. + - **string** means this method returns a string. +- Utility methods and properties. **GeneratePassword** uses methods and properties that are also + defined in the **System.Web** library. + - Methods + - `System.Web.CrossSiteScriptingValidation.IsDangerousString(string s, out int matchIndex)` + - `System.Web.CrossSiteScriptingValidation.IsAtoZ(char c)` + - Properties + - `char[] punctuations`, from **System.Web.Security.Membership** + - `char[] startingChars`, from **System.Web.CrossSiteScriptingValidation** + +Now enough C#, let get to scripting. + +### Main function + +For this, we are going to use the **Advanced Function** template, from Visual Studio Code. I'll name +the main function `New-StrongPassword`, but you can name it as you like, just remember using +approved verbs. + +This method takes as parameter two integer numbers, let's create them in the `param()` block. The +first two `if` statements are checks to ensure both parameters are within acceptable range. We can +accomplish the same with parameter attributes. + +```powershell +function New-StrongPassword { + + [CmdletBinding()] + param ( + + # Number of characters. + [Parameter( + Mandatory, + Position = 0, + HelpMessage = 'The number of characters the password should have.' + )] + [ValidateRange(1, 128)] + [int] $Length, + + # Number of non alpha-numeric chars. + [Parameter( + Mandatory, + Position = 1, + HelpMessage = 'The number of non alpha-numeric characters the password should contain.' + )] + [ValidateScript({ + if ($PSItem -gt $Length -or $PSItem -lt 0) { + $newObjectSplat = @{ + TypeName = 'System.ArgumentException' + ArgumentList = 'Membership minimum required non alpha-numeric characters is incorrect' + } + throw New-Object @newObjectSplat + } + return $true + })] + [int] $NumberOfNonAlphaNumericCharacters + + ) + + begin { + + } + + process { + + } + + end { + + } +} +``` + +### Utilities + +Now let's focus on the `Begin{}` block, and create those utility methods, and properties. + +#### Properties + +These are the two properties, in our case variables, that we need to create. + +```csharp +private static char[] startingChars = new char[2] { '<', '&' }; +private static char[] punctuations = "!@#$%^&*()_-+=[{]};:>|./?".ToCharArray(); +``` + +Let's create them as global variables, to be used across our functions if necessary. + +```powershell +[char[]]$global:punctuations = @('!', '@', '#', '$', '%', '^', '&', '*', '(', ')', '_', + '-', '+', '=', '[', '{', ']', '}', ';', ':', '>', '|', + '.', '/', '?') +[char[]]$global:startingChars = @('<', '&') +``` + +#### Get-IsAtoZ + +This is what the method looks like: + +```csharp +private static bool IsAtoZ(char c) +{ + if (c < 'a' || c > 'z') + { + if (c >= 'A') + { + return c <= 'Z'; + } + return false; + } + return true; +} +``` + +Pretty simple method, with one parameter, only the operator's name needs to change. Let's use an +inline function: + +```powershell +function Get-IsAToZ([char]$c) { + if ($c -lt 'a' -or $c -gt 'z') { + if ($c -ge 'A') { + return $c -le 'Z' + } + return $false + } + return $true +} +``` + +#### Get-IsDangerousString + +This is what the C# method looks like: + +```csharp +internal static bool IsDangerousString(string s, out int matchIndex) +{ + matchIndex = 0; + int startIndex = 0; + while (true) + { + int num = s.IndexOfAny(startingChars, startIndex); + if (num < 0) + { + return false; + } + if (num == s.Length - 1) + { + break; + } + matchIndex = num; + switch (s[num]) + { + case '<': + if (IsAtoZ(s[num + 1]) || s[num + 1] == '!' || s[num + 1] == '/' || s[num + 1] == '?') + { + return true; + } + break; + case '&': + if (s[num + 1] == '#') + { + return true; + } + break; + } + startIndex = num + 1; + } + return false; +} +``` + +This one is a little more extensive, but it's pretty much only string manipulation. The interesting +part of this method though, is the parameter **matchIndex**. Note the `out` keyword, this means this +parameter is passed as reference. We could skip this parameter altogether, because is not used in +our case, but this is a perfect opportunity to exercise the **PSReference** type. + +```powershell +function Get-IsDangerousString { + + param([string]$s, [ref]$matchIndex) + + # To access the referenced parameter's value, we use the 'Value' property from PSReference. + $matchIndex.Value = 0 + $startIndex = 0 + + while ($true) { + $num = $s.IndexOfAny($global:startingChars, $startIndex) + if ($num -lt 0) { + return $false + } + if ($num -eq $s.Length - 1) { + break + } + $matchIndex.Value = $num + + switch ($s[$num]) { + '<' { + if ( + (Get-IsAToZ($s[$num + 1])) -or + ($s[$num + 1] -eq '!') -or + ($s[$num + 1] -eq '/') -or + ($s[$num + 1] -eq '?') + ) { + return $true + } + } + '&' { + if ($s[$num + 1] -eq '#') { + return $true + } + } + } + $startIndex = $num + 1 + } + return $false +} +``` + +With these, our `Begin{}` block looks like this: + +```powershell +Begin { + [char[]]$global:punctuations = @('!', '@', '#', '$', '%', '^', '&', '*', '(', ')', '_', + '-', '+', '=', '[', '{', ']', '}', ';', ':', '>', '|', + '.', '/', '?') + [char[]]$global:startingChars = @('<', '&') + + function Get-IsAToZ([char]$c) { + if ($c -lt 'a' -or $c -gt 'z') { + if ($c -ge 'A') { + return $c -le 'Z' + } + return $false + } + return $true + } + + function Get-IsDangerousString { + + param([string]$s, [ref]$matchIndex) + + $matchIndex.Value = 0 + $startIndex = 0 + + while ($true) { + $num = $s.IndexOfAny($global:startingChars, $startIndex) + if ($num -lt 0) { + return $false + } + if ($num -eq $s.Length - 1) { + break + } + $matchIndex.Value = $num + + switch ($s[$num]) { + '<' { + if ( + (Get-IsAToZ($s[$num + 1])) -or + ($s[$num + 1] -eq '!') -or + ($s[$num + 1] -eq '/') -or + ($s[$num + 1] -eq '?') + ) { + return $true + } + } + '&' { + if ($s[$num + 1] -eq '#') { + return $true + } + } + } + $startIndex = $num + 1 + } + return $false + } +} +``` + +### Main Function Body + +In this stage we build the function itself. Since we're using attributes to check the parameters, +the first two `if` statements are ignored. After that, we have a single `do-while` loop. In this +loop, we are going to use tools from the **System.Security.Cryptography** library, so let's import +it. + +```powershell +Add-Type -AssemblyName System.Security.Cryptography + +# If you get 'Assembly cannot be found' errors, load it with partial name instead. +[void][System.Reflection.Assembly]::LoadWithPartialName('System.Security.Cryptography') +``` + +First let's declare the variables used in the main function body, and inside the main loop. This +gives us the opportunity to analyze our choices. + +```powershell +# Explicitly declaring the output 'text' to match the method. We can skip this delaration. +# Same for the 'matchIndex' +$text = [string]::Empty +$matchIndex = 0 +do { + $array = New-Object -TypeName 'System.Byte[]' -ArgumentList $Length + $array2 = New-Object -TypeName 'System.Char[]' -ArgumentList $Length + $num = 0 + + # This stage could be done in 3 ways. We could use 'New-Object' and imediately call + # 'GetBytes' on it, we could use the class constructor directly, and call 'GetBytes' + # on it: [System.Security.Cryptography.RNGCryptoServiceProvider]::new().GetBytes(), + # or we could instantiate the 'RNGCryptoServiceProvider' object using one of the + # previous methods, and call 'GetBytes' on it. Since we're using PowerShell tools the + # most we can, and we want to stay true to the method, let's use the first option. + # [void] used to suppress output. + [void](New-Object -TypeName 'System.Security.Cryptography.RNGCryptoServiceProvider').GetBytes($array) + + + # Note that when passing a variable as reference to a function parameter, we need to + # cast it to 'PSReference'. The parentheses are necessary so the parameter uses the + # object, and not use it as a string. +} while ((Get-IsDangerousString -s $text -matchIndex ([ref]$matchIndex))) +``` + +Note that in our pursuit to stay true to the method's layout, we are including extra declarations. +Although this could be avoided, in some cases it helps with script readability. Plus, if you have +experience with any programming language, this will feel familiar. + +Right after that, we have a `for` loop, which will choose each character for our password. It does +this with a series of mathematical operations, and comparisons. + +```powershell +for ($i = 0; $i -lt $Length; $i++) { + $num2 = [int]$array[$i] % 87 + if ($num2 -lt 10) { + $array2[$i] = [char](48 + $num2) + continue + } + if ($num2 -lt 36) { + $array2[$i] = [char](65 + $num2 - 10) + continue + } + if ($num2 -lt 62) { + $array2[$i] = [char](97 + $num2 - 36) + continue + } + $array2[$i] = $global:punctuations[$num2 - 62] + $num++ +} +``` + +The next session is going to manage our number of non-alphanumeric characters. It does that by +generating random symbol characters and replacing values in the array we filled in the previous +loop. + +```powershell +if ($num -lt $NumberOfNonAlphaNumericCharacters) { + $random = New-Object -TypeName 'System.Random' + + # Generating only the characters left to complete our parameter specification. + for ($j = 0; $j -lt $NumberOfNonAlphaNumericCharacters - $num; $j++) { + $num3 = 0 + do { + $num3 = $random.Next(0, $Length) + } while (![char]::IsLetterOrDigit($array2[$num3])) + $array2[$num3] = $global:punctuations[$random.Next(0, $global:punctuations.Length)] + } +} +``` + +Now all that's left is to create a string from the character array, and check if it's safe with +`Get-IsDangerousString`. + +```powershell +$text = [string]::new($array2) +``` + +If our `text` is safe, we return it and the function reaches end of execution. Our finished function +looks like this: + +```powershell +function New-StrongPassword { + + [CmdletBinding()] + param ( + + # Number of characters. + [Parameter( + Mandatory, + Position = 0, + HelpMessage = 'The number of characters the password should have.' + )] + [ValidateRange(1, 128)] + [int] $Length, + + # Number of non alpha-numeric chars. + [Parameter( + Mandatory, + Position = 1, + HelpMessage = 'The number of non alpha-numeric characters the password should contain.' + )] + [ValidateScript({ + if ($PSItem -gt $Length -or $PSItem -lt 0) { + $newObjectSplat = @{ + TypeName = 'System.ArgumentException' + ArgumentList = 'Membership minimum required non alpha-numeric characters is incorrect' + } + throw New-Object @newObjectSplat + } + })] + [int] $NumberOfNonAlphaNumericCharacters + + ) + + Begin { + [char[]]$global:punctuations = @('!', '@', '#', '$', '%', '^', '&', '*', '(', ')', '_', + '-', '+', '=', '[', '{', ']', '}', ';', ':', '>', '|', + '.', '/', '?') + [char[]]$global:startingChars = @('<', '&') + + function Get-IsAToZ([char]$c) { + if ($c -lt 'a' -or $c -gt 'z') { + if ($c -ge 'A') { + return $c -le 'Z' + } + return $false + } + return $true + } + + function Get-IsDangerousString { + + param([string]$s, [ref]$matchIndex) + + $matchIndex.Value = 0 + $startIndex = 0 + + while ($true) { + $num = $s.IndexOfAny($global:startingChars, $startIndex) + if ($num -lt 0) { + return $false + } + if ($num -eq $s.Length - 1) { + break + } + $matchIndex.Value = $num + + switch ($s[$num]) { + '<' { + if ( + (Get-IsAToZ($s[$num + 1])) -or + ($s[$num + 1] -eq '!') -or + ($s[$num + 1] -eq '/') -or + ($s[$num + 1] -eq '?') + ) { + return $true + } + } + '&' { + if ($s[$num + 1] -eq '#') { + return $true + } + } + } + $startIndex = $num + 1 + } + return $false + } + } + + Process { + Add-Type -AssemblyName 'System.Security.Cryptography' + + $text = [string]::Empty + $matchIndex = 0 + do { + $array = New-Object -TypeName 'System.Byte[]' -ArgumentList $Length + $array2 = New-Object -TypeName 'System.Char[]' -ArgumentList $Length + $num = 0 + [void](New-Object -TypeName 'System.Security.Cryptography.RNGCryptoServiceProvider').GetBytes($array) + + for ($i = 0; $i -lt $Length; $i++) { + $num2 = [int]$array[$i] % 87 + if ($num2 -lt 10) { + $array2[$i] = [char](48 + $num2) + continue + } + if ($num2 -lt 36) { + $array2[$i] = [char](65 + $num2 - 10) + continue + } + if ($num2 -lt 62) { + $array2[$i] = [char](97 + $num2 - 36) + continue + } + $array2[$i] = $global:punctuations[$num2 - 62] + $num++ + } + + if ($num -lt $NumberOfNonAlphaNumericCharacters) { + $random = New-Object -TypeName 'System.Random' + + for ($j = 0; $j -lt $NumberOfNonAlphaNumericCharacters - $num; $j++) { + $num3 = 0 + do { + $num3 = $random.Next(0, $Length) + } while (![char]::IsLetterOrDigit($array2[$num3])) + $array2[$num3] = $global:punctuations[$random.Next(0, $global:punctuations.Length)] + } + } + + $text = [string]::new($array2) + } while ((Get-IsDangerousString -s $text -matchIndex ([ref]$matchIndex))) + } + + End { + return $text + } +} +``` + +### Result + +Now all that's left is to call our function: + +![New-StrongPassword][05] + +## Conclusion + +I hope you had as much fun as I had building this function. With this new skill, you can improve +your scripts' complexity and reliability. This also makes you more comfortable to write your own +modules, binary or not. + +Thank you for going along. + +Happy scripting! + +## Links + +- [ILSpy GitHub page][08] +- [Test our WindowsUtils module!][07] +- [See what I'm up to][06] + +<!-- link references --> +[01]: ./Media/Porting-GeneratePassword-From-Csharp/File-OpenFromGAC.png +[02]: ./Media/Porting-GeneratePassword-From-Csharp/GeneratePasswordMethod.png +[03]: ./Media/Porting-GeneratePassword-From-Csharp/MembershipClass.png +[04]: ./Media/Porting-GeneratePassword-From-Csharp/OpenFromGACMenu.png +[05]: ./Media/Porting-GeneratePassword-From-Csharp/Result.png +[06]: https://github.com/FranciscoNabas +[07]: https://github.com/FranciscoNabas/WindowsUtils +[08]: https://github.com/icsharpcode/ILSpy +[09]: https://www.microsoft.com/store/productId/9MXFBKFVSQ13 diff --git a/Posts/2023/06/Measuring-Download-Time.md b/Posts/2023/06/Measuring-Download-Time.md new file mode 100644 index 00000000..cc81821a --- /dev/null +++ b/Posts/2023/06/Measuring-Download-Time.md @@ -0,0 +1,539 @@ +--- +post_title: 'Measuring average download time' +username: francisconabas +categories: PowerShell +post_slug: measuring-download-time +tags: PowerShell, Automation, Performance, Measure-Command +summary: This post shows how to measure average download time with PowerShell +--- + +One of the most overlooked roles of a systems administrator is to be able to troubleshoot +network issues. How many times had you been in a situation where your servers are problematic, +and someone asked you to check the network connectivity? +One of the steps is checking downloading time, and speed, and although there are countless +tools available, today we will learn how to do it natively, with PowerShell. + +## Methods + +We will focus on three methods, ranging from the easiest to the most complex, and discuss +their pros and cons. These methods are the `Start-BitsTransfer` Cmdlet, using .NET with the +`System.Net` namespace, and using the Windows native API. + +### Start-BitsTransfer + +BITS, or Background Intelligent Transfer Service is a Windows service that manages content transfer +using HTTP or SMB. It was designed to manage the many aspects of content transfer, including cost, +speed, priority, etc. +For us, it also serves as an easy way of downloading files. Here is how you download a file from a +web server using BITS: + +```powershell +$startBitsTransferSplat = @{ + Source = 'https://www.contoso.com/Files/BitsDefinition.txt' + Destination = 'C:\BitsDefinition.txt' +} +Start-BitsTransfer @startBitsTransferSplat +``` + +Another great advantage of BITS is that it shows progress, which can be useful while +downloading big files. In our case however, we want to know how long does it take to +download a file. For this we will use a handy object of type `System.Diagnostics.Stopwatch`. + +```powershell +$stopwatch = [System.Diagnostics.Stopwatch]::new() + +$stopwatch.Start() +$startBitsTransferSplat = @{ + Source = 'https://www.contoso.com/Files/BitsDefinition.txt' + Destination = 'C:\BitsDefinition.txt' +} +Start-BitsTransfer @startBitsTransferSplat +$stopwatch.Stop() + +Write-Output $stopwatch.Elapsed +``` + +```powershell-console +Days : 0 +Hours : 0 +Minutes : 0 +Seconds : 0 +Milliseconds : 816 +Ticks : 8165482 +TotalDays : 9.45078935185185E-06 +TotalHours : 0.000226818944444444 +TotalMinutes : 0.0136091366666667 +TotalSeconds : 0.8165482 +TotalMilliseconds : 816.5482 +``` + +Awesome, we now have a baseline to build our script upon. First thing we will change is the file. +Since we are more interested on the speed we can use temporary files to download. That also +gives us the opportunity of cleaning up at the end. For this we will use a static method from +`System.IO.Path` called `GetTempFileName`. Other thing we must think is on running the test a number +of times, and calculating the average, this way we have more reliable results. + +```powershell +# Changing the progress preference to hide the progress bar. +$ProgressPreference = 'SilentlyContinue' +$payloadUrl = 'https://www.contoso.com/Files/BitsDefinition.txt' +$stopwatch = New-Object -TypeName 'System.Diagnostics.Stopwatch' +$elapsedTime = [timespan]::Zero +$iterationNumber = 3 + +# Here we are using a foreach loop with a range, +# but this can also be accomplished with a for loop. +foreach ($iteration in 1..$iterationNumber) { + $tempFilePath = [System.IO.Path]::GetTempFileName() + + $stopwatch.Restart() + Start-BitsTransfer -Source $payloadUrl -Destination $tempFilePath + $stopwatch.Stop() + + Remove-Item -Path $tempFilePath + $elapsedTime = $elapsedTime.Add($stopwatch.Elapsed) +} + +# Timespan.Divide is not available on .NET Framework. +if ($PSVersionTable.PSVersion -ge [version]'6.0') { + $average = $elapsedTime.Divide($IterationNumber) +} else { $ + average = [timespan]::new($elapsedTime.Ticks / $IterationNumber) +} + +return $average +``` + +Great, now we can run the test as many times as we want and get consistent results. This looping +system will also serve as a skeleton for the other methods we will try. + +### System.Net.HttpWebRequest + +Using `Start-BitsTransfer` is great because it's easy to set up, however is not the most efficient +way. BITS transfers have some overhead involved to start, maintain and cleanup jobs, manage +throttling, etc. If we want to keep our results as true as possible we need to go down in the +abstraction level. +This method uses the following workflow: + +- Creates a **request** to the destination URI. +- Gets the **response**, and **response stream**. +- Creates the temporary file by opening a **file stream**. +- Downloads the binary data, and writes in the file stream. +- Closes the request, and file streams. + +Here is what this implementation looks like: + +```powershell +$uri = [uri]'https://www.contoso.com/Files/BitsDefinition.txt' + +$stopwatch = [System.Diagnostics.Stopwatch]::new() + +$request = [System.Net.HttpWebRequest]::Create($uri) + +# If necessary you can set the download timeout in milliseconds. +$request.Timeout = 15000 + +$stopwatch.Restart() + +# Receiving the first request, opening a file memory stream, and creating a buffer. +$responseStream = $request.GetResponse().GetResponseStream() +$tempFilePath = [System.IO.Path]::GetTempFileName() + +$targetStream = [System.IO.FileStream]::new($tempFilePath, 'Create') + +# You can experiment with the size of the byte array to try to get the best performance. +$buffer = [System.Byte[]]::new(10Kb) + +# Reading data and writing to the file stream, until there is no more data to read. +do { + $count = $responseStream.Read($buffer, 0, $buffer.Length) + $targetStream.Write($buffer, 0, $count) + +} while ($count -gt 0) + +# Stopping the stopwatch, and storing the elapsed time. +$stopwatch.Stop() + +# Disposing of unmanaged resources, and deleting the temp file. +$targetStream.Dispose() +$responseStream.Dispose() + +Remove-Item -Path $tempFilePath + +return $stopwatch.Elapsed +``` + +There are definitely more steps, and more points of failure, so how does it perform against +the BITS method? Here are the results of both methods, using the same file and 10 iterations. + +**BITS**: + +```powershell-console +Days : 0 +Hours : 0 +Minutes : 0 +Seconds : 0 +Milliseconds : 657 +Ticks : 6575274 +TotalDays : 7.61027083333333E-06 +TotalHours : 0.0001826465 +TotalMinutes : 0.01095879 +TotalSeconds : 0.6575274 +TotalMilliseconds : 657.5274 +``` + +**HttpWebRequest**: + +```powershell-console +Days : 0 +Hours : 0 +Minutes : 0 +Seconds : 0 +Milliseconds : 315 +Ticks : 3151956 +TotalDays : 3.64809722222222E-06 +TotalHours : 8.75543333333333E-05 +TotalMinutes : 0.00525326 +TotalSeconds : 0.3151956 +TotalMilliseconds : 315.1956 +``` + +Looking good, a little less than half. Now we know we are closer to the real time spent downloading +the file. But the question is, if .NET it's also an abstraction layer, how low can we go? +The operating system, of course. + +### Native + +Although there are multiple abstraction layers on the OS itself, +there is a user-mode API defined in `Winhttp.dll` who's exported functions can be used in PowerShell +through Platform Invoke. This means, we need to use **C#** to create these function signatures in +managed .NET. Here is what that code looks like: + +```csharp +namespace Utilities +{ + using System; + using System.IO; + using System.Runtime.InteropServices; + + public class WinHttp + { + [DllImport("Winhttp.dll", SetLastError = true, CharSet = CharSet.Unicode)] + public static extern IntPtr WinHttpOpen( + string pszAgentW, + uint dwAccessType, + string pszProxyW, + string pszProxyBypassW, + uint dwFlags + ); + + [DllImport("Winhttp.dll", SetLastError = true, CharSet = CharSet.Unicode)] + public static extern IntPtr WinHttpConnect( + IntPtr hSession, + string pswzServerName, + uint nServerPort, + uint dwReserved + ); + + [DllImport("Winhttp.dll", SetLastError = true, CharSet = CharSet.Unicode)] + public static extern IntPtr WinHttpOpenRequest( + IntPtr hConnect, + string pwszVerb, + string pwszObjectName, + string pwszVersion, + string pwszReferrer, + string ppwszAcceptTypes, + uint dwFlags + ); + + [DllImport("Winhttp.dll", SetLastError = true, CharSet = CharSet.Unicode)] + public static extern bool WinHttpSendRequest( + IntPtr hRequest, + string lpszHeaders, + uint dwHeadersLength, + IntPtr lpOptional, + uint dwOptionalLength, + uint dwTotalLength, + UIntPtr dwContext + ); + + [DllImport("Winhttp.dll", SetLastError = true, CharSet = CharSet.Unicode)] + public static extern bool WinHttpReceiveResponse( + IntPtr hRequest, + IntPtr lpReserved + ); + + [DllImport("Winhttp.dll", SetLastError = true, CharSet = CharSet.Unicode)] + public static extern bool WinHttpQueryDataAvailable( + IntPtr hRequest, + out uint lpdwNumberOfBytesAvailable + ); + + [DllImport("Winhttp.dll", SetLastError = true, CharSet = CharSet.Unicode)] + public static extern bool WinHttpReadData( + IntPtr hRequest, + IntPtr lpBuffer, + uint dwNumberOfBytesToRead, + out uint lpdwNumberOfBytesRead + ); + + [DllImport("Winhttp.dll", SetLastError = true, CharSet = CharSet.Unicode)] + public static extern bool WinHttpCloseHandle(IntPtr hInternet); + } +} +``` + +Then we can use `Add-Type` to compile, and import this type in PowerShell. + +```powershell +Add-Type -TypeDefinition (Get-Content -Path 'C:\WinHttpHelper.cs' -Raw) +``` + +After that, the method is similar to the .NET one, with a few more steps. +It makes sense being alike, because at some point .NET will call a Windows API. Note that +`Winhttp.dll` is not the only API that can be used to download files. This is what the PowerShell +code looks like: + +```powershell +$stopwatch = New-Object -TypeName 'System.Diagnostics.Stopwatch' + +# Here we open a WinHttp session, connect to the destination host, +#and open a request to the file. +$hSession = [Utilities.WinHttp]::WinHttpOpen('NativeDownload', 0, '', '', 0) +$hConnect = [Utilities.WinHttp]::WinHttpConnect($hSession, $Uri.Host, 80, 0) +$hRequest = [Utilities.WinHttp]::WinHttpOpenRequest( + $hConnect, 'GET', $Uri.AbsolutePath, '', '', '', 0 +) + +$stopwatch.Start() +# Sending the first request. +$boolResult = [Utilities.WinHttp]::WinHttpSendRequest( + $hRequest, '', 0, [IntPtr]::Zero, 0, 0, [UIntPtr]::Zero +) +if (!$boolResult) { + Write-Error 'Failed sending request.' +} +if (![Utilities.WinHttp]::WinHttpReceiveResponse($hRequest, [IntPtr]::Zero)) { + Write-Error 'Failed receiving response.' +} + +# Creating the temp file memory stream. +$tempFilePath = [System.IO.Path]::GetTempFileName() +$fileStream = [System.IO.FileStream]::new($tempFilePath, 'Create') + +# Reading data until there is no more data available. +do { + # Querying if there is data available. + $dwSize = 0 + if (![Utilities.WinHttp]::WinHttpQueryDataAvailable($hRequest, [ref]$dwSize)) { + Write-Error 'Failed querying for available data.' + } + + # Allocating memory, and creating the byte array who will hold the managed data. + $chunk = New-Object -TypeName "System.Byte[]" -ArgumentList $dwSize + $buffer = [System.Runtime.InteropServices.Marshal]::AllocHGlobal($dwSize) + + # Reading the data. + try { + $boolResult = [Utilities.WinHttp]::WinHttpReadData( + $hRequest, $buffer, $dwSize, [ref]$dwSize + ) + if (!$boolResult) { + Write-Error 'Failed to read data.' + } + + # Copying the data from the unmanaged pointer to the managed byte array, + # then ing the data into the file stream. + [System.Runtime.InteropServices.Marshal]::Copy($buffer, $chunk, 0, $chunk.Length) + $fileStream.Write($chunk, 0, $chunk.Length) + } + finally { + # Freeing the unmanaged memory. + [System.Runtime.InteropServices.Marshal]::FreeHGlobal($buffer) + } + +} while ($dwSize -gt 0) +$stopwatch.Stop() + +# Closing the unmanaged handles. +[void][Utilities.WinHttp]::WinHttpCloseHandle($hRequest) +[void][Utilities.WinHttp]::WinHttpCloseHandle($hConnect) +[void][Utilities.WinHttp]::WinHttpCloseHandle($hSession) + +# Disposing of the file stream will close the file handle, which will allow us +# to manage the file later. +$fileStream.Dispose() +$fileStream.Dispose() + +Remove-Item -Path $tempFilePath + +return $stopwatch.Elapsed +``` + +Now with all this extra work you might be asking, how does it perform? + +**HttpWebRequest**: + +```powershell-console +Days : 0 +Hours : 0 +Minutes : 0 +Seconds : 0 +Milliseconds : 281 +Ticks : 2819990 +TotalDays : 3.26387731481481E-06 +TotalHours : 7.83330555555556E-05 +TotalMinutes : 0.00469998333333333 +TotalSeconds : 0.281999 +TotalMilliseconds : 281.999 +``` + +**Native**: + +```powershell-console +Days : 0 +Hours : 0 +Minutes : 0 +Seconds : 0 +Milliseconds : 249 +Ticks : 2497170 +TotalDays : 2.89024305555556E-06 +TotalHours : 6.93658333333333E-05 +TotalMinutes : 0.00416195 +TotalSeconds : 0.249717 +TotalMilliseconds : 249.717 +``` + +Wait, that's almost the same thing, why is that? We are calling the OS API directly! +Well, we are, but we are managing everything from PowerShell, while .NET is using compiled +code, from a library. So what if we add all the request work in our C# code, and use it as a method? +Here's what said method looks like: + +```csharp +public static string NativeDownload(Uri uri) +{ + IntPtr hInternet = WinHttpOpen("NativeFileDownloader", 0, "", "", 0); + if (hInternet == IntPtr.Zero) + throw new SystemException(Marshal.GetLastWin32Error().ToString()); + + IntPtr hConnect = WinHttpConnect(hInternet, uri.Host, 443, 0); + if (hConnect == IntPtr.Zero) + throw new SystemException(Marshal.GetLastWin32Error().ToString()); + + IntPtr hReq = WinHttpOpenRequest(hConnect, "GET", uri.AbsolutePath, "", "", "", 0); + if (hReq == IntPtr.Zero) + throw new SystemException(Marshal.GetLastWin32Error().ToString()); + + if (!WinHttpSendRequest(hReq, "", 0, IntPtr.Zero, 0, 0, UIntPtr.Zero)) + throw new SystemException(Marshal.GetLastWin32Error().ToString()); + + if (!WinHttpReceiveResponse(hReq, IntPtr.Zero)) + throw new SystemException(Marshal.GetLastWin32Error().ToString()); + + string tempFilePath = Path.GetTempFileName(); + FileStream fileStream = new FileStream(tempFilePath, FileMode.Create); + uint dwBytes; + do + { + if (!WinHttpQueryDataAvailable(hReq, out dwBytes)) + throw new SystemException(Marshal.GetLastWin32Error().ToString()); + + byte[] chunk = new byte[dwBytes]; + IntPtr buffer = Marshal.AllocHGlobal((int)dwBytes); + try + { + if (!WinHttpReadData(hRequest, buffer, dwBytes, out _)) + throw new SystemException(Marshal.GetLastWin32Error().ToString()); + + Marshal.Copy(buffer, chunk, 0, chunk.Length); + fileStream.Write(chunk, 0, chunk.Length); + } + finally + { + Marshal.FreeHGlobal(buffer); + } + } while (dwBytes > 0); + + WinHttpCloseHandle(hReq); + WinHttpCloseHandle(hConnect); + WinHttpCloseHandle(hInternet); + + fileStream.Dispose(); + + return tempFilePath; +} +``` + +The results: + +```powershell-console +Days : 0 +Hours : 0 +Minutes : 0 +Seconds : 0 +Milliseconds : 191 +Ticks : 1917438 +TotalDays : 2.21925694444444E-06 +TotalHours : 5.32621666666667E-05 +TotalMinutes : 0.00319573 +TotalSeconds : 0.1917438 +TotalMilliseconds : 191.7438 +``` + +And there we go, a slighter faster download, is the small improvement worth all the extra work? +I say yes, that gives us the opportunity to expand our Operating System knowledge. + +## Bonus + +Before we wrap up, we have calculated the average time, but what about the speed? How can my script +be as cool as those internet speed measuring websites? Well, We have the time, all we need is the +file size, and we can calculate the speed: + +```powershell +$uri = [uri]'https://www.contoso.com/Files/BitsDefinition.txt' + +# Getting the total file size in bytes. +$totalSizeBytes = [System.Net.HttpWebRequest]::Create($uri).GetResponse().ContentLength + +# Elapsed time here is the result of the previous methods. +if ($Host.Version -ge [version]'6.0') { $average = $elapsedTime.Divide($IterationNumber) } +else { $average = [timespan]::new($elapsedTime.Ticks / $IterationNumber) } + +# Calculating the speed in Bytes/second +$bytesPerSecond = $totalSizeBytes / $average.TotalSeconds + +# Creating an output string based on the B/s result. +switch ($bytesPerSecond) { + { $_ -gt 99 } { $speed = "$([Math]::Round($bytesPerSecond / 1KB, 2)) Kb/s" } + { $_ -gt 101376 } { $speed = "$([Math]::Round($bytesPerSecond / 1MB, 2)) Mb/s" } + { $_ -gt 103809024 } { $speed = "$([Math]::Round($bytesPerSecond / 1GB, 2)) Gb/s" } + { $_ -gt 106300440576 } { $speed = "$([Math]::Round($bytesPerSecond / 1TB, 2)) Tb/s" } + Default { $speed = "$([Math]::Round($bytesPerSecond, 2)) B/s" } +} + +return [PSCustomObject]@{ + Speed = $speed + TimeSpan = $average +} +``` + +```powershell-console +Speed TimeSpan +----- -------- +3.6 Mb/s 00:00:00.2070106 +``` + +## Conclusion + +If you got to this point I hope you had as much fun as I did. You can find all the code we wrote +in my [GitHub page][01]. + +Until the next one, happy scripting! + +- [Test our WindowsUtils module!][02] +- [See what I'm up to][03] + +<!-- link references --> +[01]: https://github.com/FranciscoNabas/PowerShellPublic/blob/main/Get-DownloadAverageTimeAndSpeed.ps1 +[02]: https://github.com/FranciscoNabas/WindowsUtils +[03]: https://github.com/FranciscoNabas diff --git a/Posts/2023/07/Changing-Console-Title.md b/Posts/2023/07/Changing-Console-Title.md new file mode 100644 index 00000000..3fbc2f25 --- /dev/null +++ b/Posts/2023/07/Changing-Console-Title.md @@ -0,0 +1,90 @@ +--- +post_title: 'Changing your console window title' +username: FranciscoNabas +categories: PowerShell +post_slug: changing-console-title +tags: PowerShell, Automation, console, terminal +summary: This post shows how to change the title of your console terminal window. +--- + +As our skill as a PowerShell developer grows, and the complexity of our scripts increase, we start +incorporating new elements to improve the user experience. That might include changing fonts, the +background color, or the console window title. This task was already discussed in a blog post from +2004, [Can I Change the Command Window Title When Running a Script?][01]. However, the post uses VB +script, and changes the title if you are willing to open a new console. Today we learn how to do it +with PowerShell, using the same window. + +## Methods + +We will explore two ways of changing the console window title. + +- The `$Host` automatic variable. +- Console virtual terminal sequences. + +## The $Host automatic variable + +This variable contains an object that represents the current host application for PowerShell. This +object contains a property called `$Host.UI.RawUI` that allows us to change various aspects of the +current PowerShell host, including the window title. Here is how we do it. + +```powershell +$Host.UI.RawUI.WindowTitle = 'MyCoolWindowTitle!' +``` + +And with just a property value change our window title changed. + +![RawUI.WindowTitle](./Media/WindowTitle.png) + +For as simple and straight forward the previous method is, there is something to keep in mind. The +`$Host` automatic variable is host dependent. + +## Virtual terminal sequences + +Console virtual terminal sequences are control character sequences that can control various aspects +of the console when written to the output stream. The terminal sequences are intercepted by the +console host when written into the output stream. To see all sequences, and more in-depth examples +go to the [Microsoft documentation page][02]. Virtual terminal sequences are preferred because they +follow a well-defined standard, and are fully documented. The window title is limited to 255 +characters. + +To change the window title the sequence is `ESC]0;<string><ST>` or `ESC]2;<string><ST>`, where + +- `ESC` is character 0x1B. +- `<ST>` is the string terminator, which in this case is the "Bell" character 0x7. + +The bell character can also be used with the escape sequence `\a`. Here is how we change a console +window title with virtual terminal sequences. + +```powershell +$title = 'Title with terminal sequences!' + +Write-Host "$([char]0x1B)]0;$title$([char]0x7)" + +# Using the escape sequence. +Write-Host "$([char]0x1B)]0;$title`a" +``` + +## Conclusion + +PowerShell is a versatile tool that often provides multiple ways of achieving the same goal. I hope +you had as much fun reading as I had writing. See you in the next one. + +Happy scripting! + +Useful links: + +- [PowerShell automatic variable][03] +- [xterm terminal emulator][05] +- [Escape sequences][06] + +Test our PowerShell module: + +- [WindowsUtils][07] + +<!-- link references --> +[01]: https://devblogs.microsoft.com/scripting/can-i-change-the-command-window-title-when-running-a-script/ +[02]: https://learn.microsoft.com/windows/console/console-virtual-terminal-sequences +[03]: https://learn.microsoft.com/powershell/module/microsoft.powershell.core/about/about_automatic_variables#home +[05]: https://invisible-island.net/xterm/ctlseqs/ctlseqs.html +[06]: https://learn.microsoft.com/cpp/c-language/escape-sequences +[07]: https://github.com/FranciscoNabas/WindowsUtils diff --git a/Posts/2023/07/Media/WindowTitle.png b/Posts/2023/07/Media/WindowTitle.png new file mode 100644 index 00000000..f3ecb4a3 Binary files /dev/null and b/Posts/2023/07/Media/WindowTitle.png differ diff --git a/Posts/2023/11/automate-text-summarization-with-openai-powershell.md b/Posts/2023/11/automate-text-summarization-with-openai-powershell.md new file mode 100644 index 00000000..5e868432 --- /dev/null +++ b/Posts/2023/11/automate-text-summarization-with-openai-powershell.md @@ -0,0 +1,136 @@ +--- +post_title: 'Automate Text Summarization with OpenAI and PowerShell' +username: thepiyush13 +categories: PowerShell, OpenAI, Scripting +post_slug: automate-text-summarization-with-openai-powershell +tags: PowerShell, OpenAI, GPT-3.5, API, Summarization +summary: This easy-to-follow guide shows you how to use PowerShell to summarize text using OpenAI's GPT-3.5 API. +--- + +Automating tasks is the core of PowerShell scripting. Adding artificial intelligence into the mix +takes automation to a whole new level. Today, we'll simplify the process of connecting to OpenAI's +powerful text summarization API from PowerShell. Let's turn complex AI interaction into a +straightforward script. + +To follow this guide, you'll need an OpenAI API key. If you don't already have one, you'll need to +create an [OpenAI account][01] or [sign in][02] to an existing one. Next, navigate to the +[API key page][03] and create a new secret key to use. + +## Step-by-Step Function Creation + +### Step 1: Define the Function and Parameters + +We'll start by setting up our function with parameters such as the API key and text to summarize: + +```powershell +function Invoke-OpenAISummarize { + param( + [string]$apiKey, + [string]$textToSummarize, + [int]$maxTokens = 60, + [string]$engine = 'davinci' + ) + # You can add or remove parameters as per your requirements +} +``` + +### Step 2: Set Up API Connection Details + +Next, we'll prepare our connection to OpenAI's API by specifying the URL and headers: + +```powershell + $uri = "https://api.openai.com/v1/engines/$engine/completions" + $headers = @{ + 'Authorization' = "Bearer $apiKey" + 'Content-Type' = 'application/json' + } +``` + +### Step 3: Construct the Body of the Request + +We need to tell the API what we want it to do: summarize text. We do this in the request body: + +```powershell + $body = @{ + prompt = "Summarize the following text: `"$textToSummarize`"" + max_tokens = $maxTokens + n = 1 + } | ConvertTo-Json +``` + +### Step 4: Make the API Request and Return the Summary + +The final part of the function sends the request and then gets the summary back from the API: + +```powershell + $parameters = @{ + Method = 'POST' + URI = $uri + Headers = $headers + Body = $body + ErrorAction = 'Stop' + } + + try { + $response = Invoke-RestMethod @parameters + return $response.choices[0].text.Trim() + } catch { + Write-Error "Failed to invoke OpenAI API: $_" + return $null + } +} +``` + +## Running the Function + +Now, to use the function, you just need two pieces of information: your OpenAI API key and the text +to summarize. + +```powershell +$summary = Invoke-OpenAISummarize -apiKey 'Your_Key' -textToSummarize 'Your text...' +Write-Output "Summary: $summary" +``` + +Replace `'Your__Key'` with your actual key and `'Your text...'` with what you want to summarize. + +Here's a how I am running this function in my local PowerShell prompt, I copied +the text from Wikipedia: + +```powershell +$summary = Invoke-OpenAISummarize -apiKey '*********' -textToSummarize @' +PowerShell is a task automation and configuration management program from +Microsoft, consisting of a command-line shell and the associated scripting +language. Initially a Windows component only, known as Windows PowerShell, +it was made open-source and cross-platform on August 18, 2016, with the +introduction of PowerShell Core.[5] The former is built on the .NET Framework, +the latter on .NET (previously .NET Core). +'@ +``` + +and I get the following result: + +``` +PowerShell, initially Windows-only, is a Microsoft automation tool that became +cross-platform as open-source PowerShell Core, transitioning from .NET Framework +to .NET. +``` + +## Conclusion + +Combining AI with PowerShell scripting is like giving superpowers to your computer. By breaking +down each step and keeping it simple, you can see how easy it is to automate text summarization +using OpenAI's GPT-3.5 API. Now, try it out and see how you can make this script work for you! + +Remember, the beauty of scripts is in their flexibility, so feel free to tweak and expand the +function to fit your needs. + +Happy scripting and enjoy the power of AI at your fingertips! + +## References + +- [OpenAI API reference documentation][04] + +[01]: https://platform.openai.com/signup +[02]: https://platform.openai.com/login +[03]: https://platform.openai.com/account/api-keys +[04]: https://platform.openai.com/docs/api-reference diff --git a/Posts/2023/11/powershell-twilio-contact-tracing-communication.md b/Posts/2023/11/powershell-twilio-contact-tracing-communication.md new file mode 100644 index 00000000..82437d81 --- /dev/null +++ b/Posts/2023/11/powershell-twilio-contact-tracing-communication.md @@ -0,0 +1,139 @@ +--- +post_title: 'Using PowerShell and Twilio API for Efficient Communication in Contact Tracing' +username: will2win4u +categories: PowerShell, Twilio, Communication Technology +post_slug: powershell-twilio-contact-tracing-communication +tags: PowerShell, Twilio, API, Communication Technology, Contact Tracing +summary: Learn to integrate PowerShell with Twilio API and streamline communication for COVID-19 contact tracing initiatives. +--- + +The COVID-19 pandemic has underscored the importance of rapid and reliable communication technology. +One vital application is in contact tracing efforts, where prompt notifications can make a +significant difference. This guide focuses on utilizing PowerShell in conjunction with the Twilio +API to establish an automated SMS notification system, an essential communication tool for contact +tracing. + +## Integrating Twilio with PowerShell + +### Registering and Preparing Twilio Credentials + +Before diving into scripting, you need to create a Twilio account. Once registered, obtain your +Account SID and Auth Token. These credentials are the keys to accessing Twilio's SMS services. Then, +choose a Twilio phone number, which will be the source of your outgoing messages. + +### PowerShell Scripting to Send SMS via Twilio + +With your Twilio environment prepared, the next step is to configure PowerShell to interact with +Twilio's API. Start by storing your Twilio credentials as environmental variables or securely within +your script, ensuring they are not exposed or hard-coded. + +```powershell +$twilioAccountSid = 'Your_Twilio_SID' +$twilioAuthToken = 'Your_Twilio_Auth_Token' +$twilioPhoneNumber = 'Your_Twilio_Number' +``` + +After the setup and with the appropriate Twilio module installed, crafting a PowerShell script to +dispatch an SMS using Twilio's API is straightforward: + +```powershell +Import-Module Twilio + +$toPhoneNumber = 'Recipient_Phone_Number' +$credential = [pscredential]:new($twilioAccountSid, + (ConvertTo-SecureString $twilioAuthToken -AsPlainText -Force)) + +# Twilio API URL for sending SMS messages +$uri = "https://api.twilio.com/2010-04-01/Accounts/$twilioAccountSid/Messages.json" + +# Preparing the payload for the POST request +$requestParams = @{ + From = $twilioPhoneNumber + To = $toPhoneNumber + Body = 'Your body text here.' +} + +$invokeRestMethodSplat = @{ + Uri = $uri + Method = 'Post' + Credential = $credential + Body = $requestParams +} + +# Using the Invoke-RestMethod command for API interaction +$response = Invoke-RestMethod @invokeRestMethodSplat +``` + +Execute the script, and if all goes as planned, you should see a confirmation of the SMS being sent. + +### Preparing Data for Automated Notifications + +Before we can automate the sending of notifications, we need to have our contact data organized and +accessible. This is typically done by creating a CSV file, which PowerShell can easily parse and +utilize within our script. + +#### Creating a CSV File + +A CSV (Comma-Separated Values) file is a plain text file that contains a list of data. For contact +tracing notifications, we can create a CSV file that holds the information of individuals who need +to receive SMS alerts. Here is an example of what the content of this CSV file might look like: + +```csv +Name,Phone +John Doe,+1234567890 +Jane Smith,+1098765432 +Alex Johnson,+1123456789 +``` + +In this simple table, each column is separated by a comma. The first row is the header, which +describes the content of each column. Subsequent rows contain the data for each person, with their +name and phone number. + +### Automating the Process for Contact Tracing + +Once manual sending is confirmed and the CSV file is ready, you can move towards automating the +process for contact tracing: + +```powershell +Import-Module Twilio + +$contactList = Import-Csv -Path 'contact_list.csv' + +# Create Twilio API credentials +$credential = [pscredential]:new($twilioAccountSid, + (ConvertTo-SecureString $twilioAuthToken -AsPlainText -Force)) + +# Twilio API URL for sending SMS messages +$uri = "https://api.twilio.com/2010-04-01/Accounts/$twilioAccountSid/Messages.json" + +foreach ($contact in $contactList) { + $requestParams = @{ + From = $twilioPhoneNumber + To = $contact.Phone + Body = "Please be informed of a potential COVID-19 exposure. Follow public health guidelines." + } + + $invokeRestMethodSplat = @{ + Uri = $uri + Method = 'Post' + Credential = $credential + Body = $requestParams + } + $response = Invoke-RestMethod @invokeRestMethodSplat + + # Log or take action based on $response as needed +} +``` + +By looping through a list of contacts and sending a personalized SMS to each, you're leveraging +automation for mass communication—a critical piece in the contact tracing puzzle. + +## Conclusion + +In this post, we've reviewed how to establish a bridge between PowerShell and Twilio's messaging API +to execute automated SMS notifications. Such integrations are at the heart of communication +technology advancements, facilitating critical public health operations like contact tracing. + +## References +- [https://www.twilio.com/docs/api](https://www.twilio.com/docs/api) +- [https://www.twilio.com/try-twilio](https://www.twilio.com/try-twilio) diff --git a/Posts/2024/02/Media/creating-a-scalable-customised-running-environment/ModuleSetup.png b/Posts/2024/02/Media/creating-a-scalable-customised-running-environment/ModuleSetup.png new file mode 100644 index 00000000..661ac0c2 Binary files /dev/null and b/Posts/2024/02/Media/creating-a-scalable-customised-running-environment/ModuleSetup.png differ diff --git a/Posts/2024/02/creating-a-scalable-customised-running-environment.md b/Posts/2024/02/creating-a-scalable-customised-running-environment.md new file mode 100644 index 00000000..e24f014a --- /dev/null +++ b/Posts/2024/02/creating-a-scalable-customised-running-environment.md @@ -0,0 +1,137 @@ +--- +post_title: 'Creating a scalable, customised running environment' +username: rod-meaney +categories: PowerShell +post_slug: creating-a-scalable-customised-running-environment +tags: PowerShell, Automation, Toolmaking, User Experience +summary: This post shows how to create an easy to support environment with all your own cmdlets. +--- + +Often people come to PowerShell as a developer looking for a simpler life, or as a support person +looking to make their life easier. Either way, we start exploring ways to encapsulate repeatable +functionality, and through PowerShell that is cmdlets. + +How to create these is defined well in [Designing PowerShell For End Users][01]. And Microsoft +obviously have pretty good documention, including [How to Write a PowerShell Script Module][02]. I +also have a few basic rules I remember wehen creating cmdlets to go along with the above posts: + +- Always use cmdlet binding. +- Call the file name the same as the cmdlet, without the dash. + +But how do you organise them and ensure that they always load. This post outlines an approach that +has worked well for me across a few different jobs, with a few iterations to get to this point. + +## Methods + +There are 2 parts to making an effective running environment + +- Ensuring all your cmdlets for a specific module will load. +- Ensuring all your modules will load. + +### Example setup + +![folder-structure][03] + +We are aiming high here. Over time your functionality will grow and this shows a structure that +allows for growth. There are 3 modules (effectively folders): `Forms`, `BusinessUtilities` and +`GeneralUtilities`. They are broken up into 2 main groupings, `my-support` and `my-utilities`. +[ps-community-blog][04] is the GitHub repository where you can find this example. + +Inside the `GenreralUtilities` folder you can see the all-important `.psm1`, with the same name as +the folder and a couple of cmdlets I have needed over the years. The `.psm1` file is a requirement +to create a module. + +## Ensuring all your cmdlets for a specific module will load + +Most descriptions of creating modules will explain that you need to either add the cmdlet into the +`.psm1`, or load the cmdlet files in the `.psm1` file. Instead, put the below in ALL your `.psm1` +module files: + +```powershell +Get-ChildItem -Path "$PSScriptRoot\*.ps1" | ForEach-Object { + . $PSScriptRoot\$($_.Name) +} +``` + +What does this do and why does it work? + +- At a high level, it iterates over the current folder, and runs every `.ps1` file as PowerShell. +- `$PSScriptRoot` is the key here, and tells running session, what the location of the current + code is. + +This means you can create cmdlets under this structure, and they will automatically load when you +start up a new PowerShell session. + +## Ensuring all your modules will load + +So, the modules are sorted. How do we make sure the modules themselves load? It's all about the +`Profile.ps1`. You will either find it or need to create it in: + +- PowerShell 5 and lower - `$HOME\Documents\WindowsPowerShell\Profile.ps1`. +- PowerShell 7 - `$HOME\Documents\PowerShell\Profile.ps1`. +- For detailed information, see [About Profiles][05]. + +So this file runs at the start of every session that is opened on your machine. I have included +both 5 and 7, as in a lot of corporate environments, 5 is all that is available, and often people +don't have access to modify their environment. With some simple code we can ensure our modules will +open. Add this into your `Profile.ps1`: + +```powershell +Write-Host "Loading Modules for Day-to-Day use" +$ErrorActionPreference = "Stop" # A safeguard for any errors + +$MyModuleDef = @{ + Utilities = @{ + Path = "C:\work\git-personal\ps-community-blog\my-utilities" + Exclude = @(".git") + } + Support = @{ + Path = "C:\work\git-personal\ps-community-blog\my-support" + Exclude = @(".git") + } +} + +foreach ($key in $MyModuleDef.Keys) { + $module = $MyModuleDef[$key] + $exclude = $module.Exclude + + $env:PSModulePath = @( + $env:PSModulePath + $module.Path + ) -join [System.IO.Path]::PathSeparator + + Get-ChildItem -Path $module.Path -Directory -Exclude $exclude | + ForEach-Object { + Write-Host "Loading Module $($_.Name) in $Key" + Import-Module $_.Name + } +} +``` + +What does this do and why does it work? + +- At a high level, it defines your module groupings, then loads your modules into the PowerShell + session. +- `$MyModuleDef` contains the reference to your module groupings, to make sure all the sub folders + are loaded as modules. +- `Exclude` is very important. You may load the code directly of your code base, so ignoring those + as modules is important. I have also put DLL's in folders in module groupings, and ignoring these + is important as well. + +Now, every time you open any PowerShell session on your machine, all your local cmdlets will be +there, ready to use with all the wonderful functionality you have created. + +## Conclusion + +Having your own PowerShell cmdlets at your fingertips with minimal overhead or thinking makes your +PowerShell experinece so very much more rewarding. It also makes it easier to do as I like to do +and start the day with my favourite mantra: + +> Lets break some stuff! + +<!-- link references --> +[01]: https://devblogs.microsoft.com/powershell-community/designing-powershell-for-end-users/ +[02]: https://learn.microsoft.com/en-us/powershell/scripting/developer/module/how-to-write-a-powershell-script-module +[03]: ./Media/creating-a-scalable-customised-running-environment/ModuleSetup.png +[04]: https://github.com/rod-meaney/ps-community-blog +[05]: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_profiles diff --git a/Posts/2024/03/Media/simple-form-development-using-powershell/JsonAndCmdlet.png b/Posts/2024/03/Media/simple-form-development-using-powershell/JsonAndCmdlet.png new file mode 100644 index 00000000..0e351ac7 Binary files /dev/null and b/Posts/2024/03/Media/simple-form-development-using-powershell/JsonAndCmdlet.png differ diff --git a/Posts/2024/03/Media/simple-form-development-using-powershell/LaunchingASimpleForm.png b/Posts/2024/03/Media/simple-form-development-using-powershell/LaunchingASimpleForm.png new file mode 100644 index 00000000..3737765c Binary files /dev/null and b/Posts/2024/03/Media/simple-form-development-using-powershell/LaunchingASimpleForm.png differ diff --git a/Posts/2024/03/simple-form-development-using-powershell.md b/Posts/2024/03/simple-form-development-using-powershell.md new file mode 100644 index 00000000..8f8ff290 --- /dev/null +++ b/Posts/2024/03/simple-form-development-using-powershell.md @@ -0,0 +1,192 @@ +--- +post_title: 'Simple form development using PowerShell' +username: rod-meaney +categories: PowerShell +post_slug: simple-form-development-using-powershell +tags: PowerShell, Automation, Toolmaking, User Experience +summary: Create .NET forms without all the .NET. +--- + +PowerShell is a tool for the command line. Most people who use it are comfortable with the command +line. But sometimes, there are valid use cases to provide Graphical User Interface (GUI). + +[alert type="important" heading="Important caveat"] +As PowerShell developers we need to be careful. We can do insanely complicated things with GUI's +(and the .NET classes), and that is not a rod we want to make for our own back! +[/alert] + +Forms are based on .NET classes, but I have implemented a _framework_, so you do nothing more than +create a JSON configuration and write simple functions in PowerShell. These functions are event +based functions contained in PowerShell cmdlets. + +I am going to break this post into 3 parts: + +- Lets just get some forms up and running +- How does all that work +- Use cases for forms and PowerShell + +## Lets just get some forms up and running + +1. Download my [ps-community-blog][01] repository. +1. If you know about PowerShell modules, add all the modules, or ALL the `ps1` files to your current + setup. If you don't, that is OK, have a quick read of + [Creating a scalable, customised running environment][02], which shows you how to set up your + PowerShell environment. The instructions in that post are actually for the same repository that + this post uses, so it should be pretty helpful. +1. Restart your current PowerShell session, which should load all the new modules. +1. In the PS terminal window, run the cmdlet. + + ```powershell + New-SampleForm + ``` + + ![launching-a-simple-form][03] + + The PS terminal window that you launch the form from is now a slave to the form you have opened. + I basically use this as an output for the user, so put it next to the opened form. If you have + made it this far, thats it! If not, review your `Profile.ps1` as suggested in + [Creating a scalable, customised running environment][02]. + +1. Press the buttons and see what happens. You should see responses appear in the PS terminal + window. The tram buttons call an API to get trams approaching stops in Melbourne, Australia for + the current time. The other two buttons are just some fun ones I found when searching for + functionality to show in the forms. + +### How do I create my own forms + +Rather than following documentation (which, lets be honest, I have not written), understanding the +basics, and copying the examples is really the quickest way. Lets look at the SampleForm and work it +through. You need a matching json and ps1 form. + +![json-and-cmdlet][04] + +I am not going to go into all the specifics, they should be obvious from the examples. But +basically, a form has a list of elements, and they are placed at an x-y coordinate based on the x-y +attribute in the element. When creating elements, the following is important: + +- Create a base json file of the right form size, with nothing in it. +- Create base matching cmdlet with only `# == TOP ==` and `# == BOTTOM ==` sections in it. These 2 + sections are identical in all form cmdlets. +- Restart your PowerShell session to pick up the new cmdlet. +- Add in elements 1 by 1 to the json file, getting them in the right position. You run the cmdlet + after making changes to the json file. +- `Important`: follow a naming convention, **type_form_specificElement**, for two reasons. + 1. Firstly you can't have the same name for an element on the form + 1. Secondly, if you start getting fancy and having tabs, including the form in the name is going + to help you immensely. (I had to do a lot of refactoring when I added in tabs!) +- Add in the `Add_Click` functions for your buttons. In keeping it simple, most of your + functionality will be driven by your buttons. After updating your cmdlets, you will need to + restart your PowerShell session to pick up the changes. I have found that using VS Code and + PowerShell plugins and restarting PowerShell sessions is much cleaner than trying to unload, and + load modules when you update/add cmdlets. + +And that is it. As a good friend/co-worker of mine says, it sounds easy when you say it quick, but +the devil is in the detail. It can also be hard to debug. + +> An easy way to debug is to create a `ps1` file with 1 line, the `New-Form` cmdlet. Running this in +> debug with breakpoints is the easiest way to debug. + +With just this, and some diving into the other examples, you will be surprised the amount of +functionality you can expose through your own GUI. + +## How does all that work + +PowerShell has access to all the .NET classes sitting underneath it and it has a rich and well +developed set of widgets to add to forms. Now I am not a .NET developer, but it is pretty intuitive. + +### Load the Assemblies and look at the base cmdlets + +Inside `GeneralUtilities.psm1` you will see: + +```powershell +Get-ChildItem -Path "$PSScriptRoot\*.ps1" | ForEach-Object{ + . $PSScriptRoot\$($_.Name) +} +Add-Type -assembly System.Windows.Forms +Add-Type -AssemblyName System.Drawing +``` + +- The first lines are my standard practice to load all the cmdlets in the module +- The `Add-Type` lines here are the crucial ones. They tell the PowerShell session to load the .NET + classes required for forms to function. +- Inside the `GeneralUtilities` module are 3 important cmdlets + - `Set-FormFromJson` is sort of the driver, reads the json file, and iterates over all the + elements, loading them onto the form by calling.. + - `Set-FormElementsFromJson` which is where all the heavy .NET lifting is done. .NET Forms have + been around so long, and are so consistent (and trust me, coming from an early 2000's web + developer, this is wonderful), that with a basic switch, you can implement them all very easily + and expose the features easily through our JSON configuration. This could be developed + infinitely more, but see the caveat at the start of this post - KISS is very important. + - `ConvertTo-HashtableV5` One of the most useful techniques in PowerShell is to always use the + native objects (hashes and lists) so that the operations are consistent. I have found this + particularly relevant for JSON files. I have included this as I rely on it heavily due to + PowerShell 5 having some deficiencies in this area. I like to have all my stuff work in + PowerShell 5 AND 7. It is based on a post [Convert JSON to a PowerShell hash table][05]. + +### Creating a form + +```powershell +function New-SampleForm { + [CmdletBinding()] + param () + # ===== TOP ===== + $FormJson = $PSCommandPath.Replace(".ps1",".json") + $NewForm, $FormElements = Set-FormFromJson $FormJson + + # ===== Single Tab ===== + # All your button clicks etc. + + # ===== BOTTOM ===== + $NewForm.ShowDialog() +} +Export-ModuleMember -Function New-SampleForm +``` + +The above is a template for creating any form. I am a firm believer of convention over +configuration. It makes for less code and simpler design. With that in mind: + +- `New-Sample` cmdlet should be in file `NewSample.ps1`. +- `NewSample.json` will be the configuration file for the form. +- The **TOP** section finds the json file for the cmdlet based on convention, then loads all the + elements. +- The **BOTTOM** section makes the form appear. +- **TOP** and **BOTTOM** sections will not change between different forms. + +Everything else in between is where the fun happens. Copy and paste `Add_Click` functions, rename +them following your JSON configuration, and you are away. + +## Use cases for forms and PowerShell + +### Quick access to common support tasks + +The support team I am involved with have gone through a maturation of using PowerShell for support +tasks over the last couple of years. We started just writing small cmdlets to do repeatable tasks. +Stuff to do with file movement, Active Directory changes, data manipulation. Next we made some +cmdlets to access vendors API's that helped us do tasks quickly instead of through the vendor GUI +application. + +All this functionality is now available through a tool that all the support guys use daily, and have +even started contributing to. + +### Postman for 'one thing' + +If you don't know Postman, it is a tool used to test API's / Web Services and is one of a modern +developers most useful tools. But we have some very technically savvy users, that are not +developers, and the ability for them to use some complex API's dramatically improves their +productivity (especially in non-production). Its too easy to make mistakes in Postman, and for +repeatable tasks with half dozen inputs, we now have a tool that does some basic validation, and +hits the API endpoint with consistent and useful data. + +## Conclusion + +You can get some big bang for minimal effort with the .NET Forms and help your fellow workers in an +environment that may just be a bit easier for some of them than native cmdlets. Sooooo... + +> Lets break some stuff! + +<!-- link references --> +[01]: https://github.com/rod-meaney/ps-community-blog +[02]: https://devblogs.microsoft.com/powershell-community/creating-a-scalable-customised-running-environment/ +[03]: ./Media/simple-form-development-using-powershell/LaunchingASimpleForm.png +[04]: ./Media/simple-form-development-using-powershell/JsonAndCmdlet.png +[05]: https://4sysops.com/archives/convert-json-to-a-powershell-hash-table/ diff --git a/Posts/2024/04/Media/encrypting-secrets-locally/KeyValueStore.png b/Posts/2024/04/Media/encrypting-secrets-locally/KeyValueStore.png new file mode 100644 index 00000000..eb5d11fd Binary files /dev/null and b/Posts/2024/04/Media/encrypting-secrets-locally/KeyValueStore.png differ diff --git a/Posts/2024/04/encrypting-secrets-locally.md b/Posts/2024/04/encrypting-secrets-locally.md new file mode 100644 index 00000000..65db5464 --- /dev/null +++ b/Posts/2024/04/encrypting-secrets-locally.md @@ -0,0 +1,145 @@ +--- +post_title: 'Encrypting secrets locally' +username: rod-meaney +categories: PowerShell +post_slug: encrypting-secrets-locally +tags: Automation, Toolmaking, Security +summary: Keeping security folks happy (or less upset which is the best we can hope for) +--- + +If you are involved in support or development, often you need to use secrets, passwords, or +subscription keys in PowerShell scripts. These need to be kept secure and separate from your scripts +but you also need access to them ALL THE TIME. + +So instead of hand entering them every time they should be stored in a key store of some sort that +you can access programmatically. Often off the shelf keystores are not available in your environment +or are clumsy to access with PowerShell. A simple way to have easy access to these secrets with +PowerShell would be helpful. + +You could simply have them in plain text, on your machine only, making it relatively secure. +However, there are many risks with this approach, so adding some additional security is an excellent +idea. + +The .NET classes sitting behind PowerShell provide some simple ways to do this. This blog will go +through + +- Basic encryption / decryption +- Using it day-to-day +- Your own form-based key store + +## Basic encryption / decryption + +The [protect][07] and [unprotect][08] methods available as part of the cryptography classes are +easy to use. However they use Byte arrays that we can simplify by wrapping their use in a String. + +The following examples can be found at the [MachineAndUserEncryption.ps1][06] module in my +[ps-community-blog][04] repository on GitHub. + +### Encryption + +```powershell +Function Protect-WithUserKey { + param( + [Parameter(Mandatory=$true)] + [string]$secret + ) + Add-Type -AssemblyName System.Security + $bytes = [System.Text.Encoding]::Unicode.GetBytes($secret) + $SecureStr = [Security.Cryptography.ProtectedData]::Protect( + $bytes, # contains data to encrypt + $null, # optional data to increase entropy + [Security.Cryptography.DataProtectionScope]::CurrentUser # scope of the encryption + ) + $SecureStrBase64 = [System.Convert]::ToBase64String($SecureStr) + return $SecureStrBase64 +} +``` + +Just going through the lines we can see + +1. PowerShell needs to know about the .NET classes (I have tested under version 5 & 7 of PowerShell) +1. We need to convert our string into a Byte array +1. Use the .NET class to encrypt +1. Convert the encrypted Byte array to a string for easy storage and retrieval +1. Return that string + +### Decryption + +```powershell +Function Unprotect-WithUserKey { + param ( + [Parameter(Mandatory=$true)] + [string]$enc_secret + ) + Add-Type -AssemblyName System.Security + $SecureStr = [System.Convert]::FromBase64String($enc_secret) + $bytes = [Security.Cryptography.ProtectedData]::Unprotect( + $SecureStr, # bytes to decrypt + $null, # optional entropy data + [Security.Cryptography.DataProtectionScope]::CurrentUser) # scope of the decryption + $secret = [System.Text.Encoding]::Unicode.GetString($bytes) + return $secret +} +``` + +Steps are identical for the decryption, using slightly different methods + +1. PowerShell needs to know about the .NET classes +1. We need to convert our string into a Byte array +1. Use the .NET class to decrypt +1. Convert the encrypted Byte array to a string +1. Return that string + +## Using it day-to-day + +This is really useful if you are doing repetitive tasks that need these values. Often in a support +role, investigations using API's can speed up the process of analysis, and also provide you with a +quick way to do fixes that don't require heavy use of a GUI based environment. + +Assigning a key to a secret value, and storing that in a hash table format is the simplest way to +have access to these values AND keep them stored locally with a degree of security. Your code can +then dynamically look up these values, and if other support people store the same key locally the +same way (often with different values, think of an API password and or username pair) then your +script can work for everyone. + +Again, `MachineAndUserEncryption.ps1` in my repository on my GitHub has functions for persisting and +using this information. For compatibility with version 5 & 7 you also need the function +[ConvertToHashtableV5][05]. + +I would also recommend using `Protect-WithMachineAndUserKey` and `Unprotect-WithMachineAndUserKey` +when implementing locally, they add another layer of protection. + +## Your own form-based key store + +If you have followed my other 2 blogs about a [scalable environment][02] and +[simple form development][03] then using the resources from these we can easily create our own form +to manage our secrets. In fact, if you have downloaded and installed the modules for either of those +blogs (they are the same, and this blog references the same as well), you have it ready to go. + +Once you have your environment set up, simply run the cmdlet: + +```powershell +New-EncryptKeyForm +``` + +and if all is set up correctly, you should see + +![key-value-secret-store][01] + +## Conclusion + +Balancing the pragmatic ease of use and security concerns around secrets you may need to use all day +every day can be a fine balancing act. Using some simple methods, we can strike that balance and +hopefully be securely productive. + +> Lets secure some stuff! + +<!-- link references --> +[01]: ./Media/encrypting-secrets-locally/KeyValueStore.png +[02]: https://devblogs.microsoft.com/powershell-community/creating-a-scalable-customised-running-environment/ +[03]: https://devblogs.microsoft.com/powershell-community/simple-form-development-using-powershell/ +[04]: https://github.com/rod-meaney/ps-community-blog +[05]: https://github.com/rod-meaney/ps-community-blog/blob/main/my-utilities/GeneralUtilities/ConvertToHashtableV5.ps1 +[06]: https://github.com/rod-meaney/ps-community-blog/blob/main/my-utilities/GeneralUtilities/MachineAndUserEncryption.ps1 +[07]: https://learn.microsoft.com/dotnet/api/system.security.cryptography.protecteddata.protect +[08]: https://learn.microsoft.com/dotnet/api/system.security.cryptography.protecteddata.unprotect \ No newline at end of file diff --git a/README.md b/README.md index 4b85edfd..4aac33aa 100644 --- a/README.md +++ b/README.md @@ -1,22 +1,69 @@ -# Community-Blog +# PowerShell Community Blog -Submissions for posts to the -[PowerShell Community blog](https://devblogs.microsoft.com/powershell-community). +This repository is for submissions for posts to the [PowerShell Community Blog][03]. We welcome +submissions to the blog from anyone in the community. -Participation in this blog is governed by the -[Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/) or the -[.NET Foundation Code of Conduct](https://dotnetfoundation.org/code-of-conduct). For more -information, see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/). +## The Purpose of This Blog -## Contributing to the blog +The intended purpose of the PowerShell Community Blog is to provide a platform for the PowerShell +Community to show off the great things you can do with PowerShell. This blog welcomes submissions to +the blog both from internal Microsoft teams and external people. Blog posts can cover the open +source PowerShell 7 or Windows PowerShell. Your post should clearly outline the problem the being +solved. Your post can cover an advanced topic, but most posts are likely to be at the 200-300 level. +For advanced topics, consider splitting your subject into a multipart series of posts. -The intended purpose of this blog is to provide a blogging platform for the PowerShell Community, -both internal and external to Microsoft's PowerShell team. Posts to the blog can discuss products -and technologies that are not part of the core PowerShell product or even made by Microsoft, as long -as the content is relevant to PowerShell users and is not marketing those products. +Posts to the blog can discuss products and technologies that aren't part of the core PowerShell +product or even made by Microsoft, as long as the post's content is relevant to PowerShell users and +is not marketing those products. -To get started, you must create a WordPress profile. Then you can create and submit your blog post. -See the [Wiki pages](https://github.com/PowerShell/Community-Blog/wiki) for detailed instructions. +The published language for the PowerShell community is English, and mainly American English, +although posts other variations of English are acceptable. The article review process focuses on the +language and structure of each article, as well as the specific details. Even if English isn't your +first language, the review process can help to iron out any problems. -Acceptance of any blog post is done at the sole discretion of the Blog admins. Before we can accept -a submission, you must sign the Contributor License Agreement (CLA). This is a one-time event. +## How to Interact + +There are several ways you can interact, depending on your needs and levels of enthusiasm. + +1. First is to read and enjoy the blog. Over time, we hope and expect the major search engines to + index these posts, making it easy for IT Pros to find and use the information contained. + +1. You can comment on any of blog posts directly on the blog. This blog uses Word Press, so in order + to add comments, you need to create and then login to a Word Press account. Once you logon + successfully to the blog, Word Press allows you to add comments to the posts here. + + To create a Word Press account, see our [Community Blog docs][06] for detailed instructions. We + welcome comments and prefer them in English. You can use an online translator like + [Bing Translator][07] if English isn't your first language. + +1. You may want to contribute your own blog post. You can create new posts, file issues on any + article (or proposed article), or help review content submissions. If you find an error, feel + free to [file an issue on GitHub][05]. You can also file an issue to suggest a specific topic you + feel might make a good blog post. + +You need a GitHub account to be able to submit anything to the blog's GitHub repository. You can +sign up for GitHub at [GitHub's new account sign up page][04]. And, you need to be able to use GitHub +and, most likely, git on your workstation. + +> [!NOTE] +> Acceptance of any blog post is done at the sole discretion of the Blog administrators. Once you +> submit a PR, the build automation adds a comment to the PR asking you to sign the CLA. The comment +> contains a link to take you to the CLA signing page. Before we can accept any blog post +> submission, you must sign the Contributor License Agreement (CLA). This is a one-time event. + +## Code of Conduct + +Please see our [Code of Conduct][01] before participating in this project. + +## Security Policy + +For any security issues, please see our [Security Policy][02]. + +<!-- link references --> +[01]: .github/CODE_OF_CONDUCT.md +[02]: .github/SECURITY.md +[03]: https://devblogs.microsoft.com/powershell-community +[04]: https://github.com/join?source=login +[05]: https://github.com/PowerShell/Community-Blog/issues +[06]: https://github.com/PowerShell/Community-Blog/tree/main/Docs +[07]: https://www.bing.com/translator