The Pipeline

Note

Learn how PowerShell's pipeline passes objects between commands, making it the most powerful feature of PowerShell.

Overview

The pipeline (|) is PowerShell's superpower. It lets you take the output of one command and send it directly to another command as input. Unlike traditional shells that pass text, PowerShell passes actual objects with properties and methods. This makes it incredibly powerful for chaining commands together to accomplish complex tasks.

Basic Concept

# Without pipeline - multiple steps
$processes = Get-Process
$filteredProcesses = Where-Object -InputObject $processes {$_.CPU -gt 100}
Format-Table -InputObject $filteredProcesses

# With pipeline - one flowing command
Get-Process | Where-Object {$_.CPU -gt 100} | Format-Table

Key Points

Pipeline passes objects, not text (this is huge!)
Use | (pipe character) to connect commands
Each command processes what the previous command outputs
Objects flow left to right through the pipeline
"Filter left, format right" - filter early for performance

Objects vs Text

PowerShell Way (Objects)

# Get process info
Get-Process | Select-Object Name, CPU, WS

# You get objects with properties you can access
$proc = Get-Process | Select-Object -First 1
$proc.Name      # Access the Name property
$proc.CPU       # Access the CPU property

Old Shell Way (Text)

# In Bash/CMD, you'd get text and have to parse it
ps aux | grep something | awk '{print $1}'  # Text manipulation

Why objects are better: - No parsing required - properties are already separated - Type-safe - numbers are numbers, dates are dates - IntelliSense/autocomplete works - Can access methods and properties directly

Basic Pipeline Usage

Simple Filtering

# Get all running services
Get-Service | Where-Object {$_.Status -eq "Running"}

# Get large files
Get-ChildItem -Path C:\Temp | Where-Object {$_.Length -gt 1MB}

# Find processes using lots of memory
Get-Process | Where-Object {$_.WS -gt 100MB}

Sorting Results

# Sort processes by CPU usage (descending)
Get-Process | Sort-Object -Property CPU -Descending

# Sort files by size
Get-ChildItem | Sort-Object -Property Length

# Sort by multiple properties
Get-Process | Sort-Object -Property CPU, WS -Descending

Selecting Properties

# Get only specific properties
Get-Process | Select-Object Name, CPU, WS

# Get first 10 items
Get-Process | Select-Object -First 10

# Get last 5 items
Get-ChildItem | Select-Object -Last 5

# Select and rename properties
Get-Process | Select-Object Name, @{Name="MemoryMB";Expression={$_.WS/1MB}}

Common Pipeline Patterns

Pattern 1: Filter → Sort → Format

# Get large files, sort by size, display as table
Get-ChildItem -Path C:\Temp |
    Where-Object {$_.Length -gt 10MB} |
    Sort-Object -Property Length -Descending |
    Format-Table Name, Length, LastWriteTime

Pattern 2: Filter → Select → Export

# Get running services, select properties, export to CSV
Get-Service |
    Where-Object {$_.Status -eq "Running"} |
    Select-Object Name, DisplayName, Status |
    Export-Csv -Path "C:\Temp\services.csv" -NoTypeInformation

Pattern 3: Get → Process → Measure

# Get log files and calculate total size
Get-ChildItem -Path C:\Logs -Filter *.log |
    Measure-Object -Property Length -Sum |
    Select-Object Count, @{Name="TotalSizeGB";Expression={$_.Sum/1GB}}

Pattern 4: Process Each Item

# Restart multiple services
"Spooler", "W32Time" |
    ForEach-Object {
        Restart-Service -Name $_ -Force
        Write-Output "Restarted $_"
    }

The Pipeline Variable: $_

In pipeline operations, $_ (or $PSItem) represents the current object:

# $_ is the current item in the pipeline
Get-Process | Where-Object {$_.CPU -gt 50}
#                            ^^^ Current process object

Get-ChildItem | ForEach-Object {
    Write-Output "File: $($_.Name), Size: $($_.Length)"
    #                    ^^^ Current file object
}

# $PSItem is the same as $_ (more readable)
Get-Service | Where-Object {$PSItem.Status -eq "Running"}

Common Pipeline Cmdlets

Where-Object (Filter)

# Filter by condition
Get-Process | Where-Object {$_.CPU -gt 100}

# Multiple conditions with -and
Get-Process | Where-Object {($_.CPU -gt 50) -and ($_.WS -gt 100MB)}

# Multiple conditions with -or
Get-Service | Where-Object {($_.Status -eq "Running") -or ($_.StartType -eq "Automatic")}

# Simplified syntax (PowerShell 3.0+)
Get-Process | Where-Object CPU -GT 100
Get-Service | Where-Object Status -EQ "Running"

ForEach-Object (Process)

# Process each item
Get-ChildItem | ForEach-Object {
    Write-Output "Processing $($_.Name)..."
}

# Perform action on each
Get-Process | ForEach-Object {
    if ($_.CPU -gt 100) {
        Write-Output "$($_.Name) is using high CPU"
    }
}

# Simplified syntax with member access
Get-Process | ForEach-Object Name  # Gets Name property of each

Select-Object (Choose Properties/Items)

# Select specific properties
Get-Process | Select-Object Name, CPU, WS

# First/Last items
Get-Process | Select-Object -First 5
Get-Process | Select-Object -Last 3

# Skip items
Get-Process | Select-Object -Skip 10 -First 5  # Skip first 10, get next 5

# Unique values
1,2,2,3,3,3,4 | Select-Object -Unique  # 1,2,3,4

# Calculated properties
Get-Process | Select-Object Name, @{Name="MemoryMB";Expression={[math]::Round($_.WS/1MB, 2)}}

Sort-Object

# Sort ascending (default)
Get-Process | Sort-Object -Property CPU

# Sort descending
Get-Process | Sort-Object -Property CPU -Descending

# Sort by multiple properties
Get-ChildItem | Sort-Object -Property Extension, Name

# Sort with custom logic
Get-Process | Sort-Object -Property @{Expression={$_.CPU}; Descending=$true}, Name

Measure-Object

# Count items
Get-ChildItem | Measure-Object
# Returns: Count

# Sum property
Get-ChildItem | Measure-Object -Property Length -Sum
# Returns: Count, Sum

# Get statistics
1..100 | Measure-Object -Average -Sum -Maximum -Minimum
# Returns: Count, Average, Sum, Maximum, Minimum

Group-Object

# Group processes by company
Get-Process | Group-Object -Property Company

# Group files by extension
Get-ChildItem | Group-Object -Property Extension

# Count occurrences
Get-Service | Group-Object -Property Status | Select-Object Name, Count

Real-World Examples

Example: Find and Clean Up Old Files

Scenario: Find log files older than 30 days and calculate space savings

$cutoffDate = (Get-Date).AddDays(-30)

# Find old log files and show total size
$oldLogs = Get-ChildItem -Path C:\Logs -Filter *.log -Recurse |
    Where-Object {$_.LastWriteTime -lt $cutoffDate} |
    Select-Object FullName, Length, LastWriteTime

# Calculate total size
$totalSize = $oldLogs |
    Measure-Object -Property Length -Sum |
    Select-Object @{Name="TotalGB";Expression={[math]::Round($_.Sum/1GB, 2)}}

Write-Output "Found $($oldLogs.Count) old log files"
Write-Output "Total size: $($totalSize.TotalGB) GB"

# Display files
$oldLogs | Format-Table FullName, @{Name="SizeMB";Expression={[math]::Round($_.Length/1MB, 2)}}, LastWriteTime

# Optional: Delete them
# $oldLogs | Remove-Item -Force -WhatIf

Example: Analyze Service Status

Scenario: Get a summary of services by status

# Group services by status and count
Get-Service |
    Group-Object -Property Status |
    Select-Object @{Name="Status";Expression={$_.Name}},
                  @{Name="Count";Expression={$_.Count}},
                  @{Name="Services";Expression={$_.Group.Name -join ', '}} |
    Format-Table -AutoSize

# Find automatic services that aren't running
$stoppedAutoServices = Get-Service |
    Where-Object {($_.StartType -eq "Automatic") -and ($_.Status -ne "Running")} |
    Select-Object Name, DisplayName, Status, StartType

if ($stoppedAutoServices) {
    Write-Warning "Found $($stoppedAutoServices.Count) automatic services that are stopped:"
    $stoppedAutoServices | Format-Table
}

Example: Process Performance Report

Scenario: Find top 10 processes by CPU and memory usage

Write-Output "`n===== Top 10 Processes by CPU ====="
Get-Process |
    Where-Object {$_.CPU -gt 0} |
    Sort-Object -Property CPU -Descending |
    Select-Object -First 10 Name, CPU, @{Name="MemoryMB";Expression={[math]::Round($_.WS/1MB, 2)}} |
    Format-Table -AutoSize

Write-Output "`n===== Top 10 Processes by Memory ====="
Get-Process |
    Sort-Object -Property WS -Descending |
    Select-Object -First 10 Name, @{Name="MemoryMB";Expression={[math]::Round($_.WS/1MB, 2)}}, CPU |
    Format-Table -AutoSize

Write-Output "`n===== Summary ====="
Get-Process |
    Measure-Object -Property WS -Sum |
    Select-Object @{Name="TotalProcesses";Expression={$_.Count}},
                  @{Name="TotalMemoryGB";Expression={[math]::Round($_.Sum/1GB, 2)}} |
    Format-List

Example: User Activity Report from Event Logs

Scenario: Find recent logon events and summarize by user

# Get logon events from last 24 hours
$startTime = (Get-Date).AddHours(-24)

Get-WinEvent -FilterHashtable @{
    LogName = 'Security'
    ID = 4624  # Successful logon
    StartTime = $startTime
} -ErrorAction SilentlyContinue |
    Select-Object TimeCreated,
                  @{Name="User";Expression={$_.Properties[5].Value}} |
    Where-Object {$_.User -notlike "*$*"} |  # Filter out system accounts
    Group-Object -Property User |
    Select-Object @{Name="Username";Expression={$_.Name}},
                  @{Name="LogonCount";Expression={$_.Count}},
                  @{Name="FirstLogon";Expression={($_.Group.TimeCreated | Measure-Object -Minimum).Minimum}},
                  @{Name="LastLogon";Expression={($_.Group.TimeCreated | Measure-Object -Maximum).Maximum}} |
    Sort-Object -Property LogonCount -Descending |
    Format-Table -AutoSize

Performance: Filter Left, Format Right

DO THIS (Fast)

# Filter FIRST (left), then format (right)
Get-Process |
    Where-Object {$_.CPU -gt 100} |      # Filter early - reduces data
    Sort-Object -Property CPU |           # Sort smaller dataset
    Select-Object -First 10 |             # Select from smaller dataset
    Format-Table                          # Format at the end

DON'T DO THIS (Slow)

# Formatting first breaks the pipeline and hurts performance
Get-Process |
    Format-Table |                        # BAD: Formatting early
    Where-Object {$_.CPU -gt 100}         # Won't work - format broke the objects

Why: - Filtering early reduces the amount of data flowing through the pipeline - Format cmdlets (Format-Table, Format-List) should always be last - Formatting converts objects to formatted output, breaking object properties

Pipeline Best Practices

Use the Pipeline for Multiple Operations

# GOOD: Pipeline chains operations
Get-ChildItem -Path C:\Temp |
    Where-Object {$_.Length -gt 1MB} |
    Sort-Object -Property Length -Descending |
    Select-Object Name, Length, LastWriteTime

# BAD: Multiple variables
$files = Get-ChildItem -Path C:\Temp
$bigFiles = $files | Where-Object {$_.Length -gt 1MB}
$sorted = $bigFiles | Sort-Object -Property Length -Descending
$selected = $sorted | Select-Object Name, Length, LastWriteTime

Break Long Pipelines into Multiple Lines

# GOOD: Readable multi-line pipeline
Get-Process |
    Where-Object {$_.CPU -gt 50} |
    Sort-Object -Property CPU -Descending |
    Select-Object -First 10 |
    Format-Table Name, CPU, WS

# BAD: Hard to read single line
Get-Process | Where-Object {$_.CPU -gt 50} | Sort-Object -Property CPU -Descending | Select-Object -First 10 | Format-Table Name, CPU, WS

Avoid Format Cmdlets in the Middle

# BAD: Format breaks the pipeline
Get-Process |
    Format-Table |
    Where-Object {$_.CPU -gt 50}  # Won't work - objects are now formatted

# GOOD: Format at the end only
Get-Process |
    Where-Object {$_.CPU -gt 50} |
    Format-Table

Tips & Tricks

Use -PassThru to Continue Pipeline After Modifying

# Without -PassThru, no output to pipeline
Get-Service -Name "Spooler" | Start-Service
# Returns nothing

# With -PassThru, object continues in pipeline
Get-Service -Name "Spooler" |
    Start-Service -PassThru |
    Select-Object Name, Status
# Can continue processing

Save Pipeline Results to Variable

# Process and save results
$topProcesses = Get-Process |
    Sort-Object -Property CPU -Descending |
    Select-Object -First 10

# Now you can reuse without re-running
$topProcesses | Export-Csv processes.csv
$topProcesses | Format-Table

Use Parentheses for Sub-Pipelines

# Get processes started by top 5 CPU processes
Get-Process -Id (
    Get-Process |
        Sort-Object CPU -Descending |
        Select-Object -First 5 -ExpandProperty Id
)

Don't Pipe to Out-File or Set-Content

# This writes formatted text, not CSV
Get-Process | Format-Table | Out-File processes.txt

# Use Export-Csv for data
Get-Process | Export-Csv processes.csv

# Or Export-Clixml to preserve objects
Get-Process | Export-Clixml processes.xml

Watch Out for Unwanted Output

# This outputs both the service object AND success message
Get-Service -Name "Spooler" |
    Start-Service |  # Returns nothing by default
    ForEach-Object { Write-Output "Started" }
# Nothing in pipeline here

# Use -PassThru if you need the object
Get-Service -Name "Spooler" |
    Start-Service -PassThru |
    ForEach-Object { Write-Output "Started $($_.Name)" }

Format Cmdlets Convert to Strings

# After Format-Table, you can't access properties
$result = Get-Process | Format-Table
$result.Name  # Won't work - it's formatted text now

# Keep objects until the end
$processes = Get-Process | Select-Object Name, CPU
$processes[0].Name  # Works - still objects
$processes | Format-Table  # Format when displaying

Arrays & Collections - Processing collections in the pipeline
Functions - Creating pipeline-aware functions
ForEach-Object - Processing items in the pipeline
Where-Object - Filtering in the pipeline
Object Manipulation - Working with pipeline objects