EDUCBA

EDUCBA

MENUMENU
  • Free Tutorials
  • Free Courses
  • Certification Courses
  • 360+ Courses All in One Bundle
  • Login
Home Data Science Data Science Tutorials PowerShell Tutorial PowerShell Invoke-Webrequest
Secondary Sidebar
PowerShell Tutorial
  • cmdlet
    • PowerShell Wait
    • PowerShell Match
    • cmdlets in PowerShell
    • Start PowerShell from cmd
    • Add-Content in PowerShell
    • Get Help in PowerShell
    • PowerShell Copy-Item
    • PowerShell Remove-Item
    • PowerShell Move-Item
    • Get Command in PowerShell
    • PowerShell Run Command
    • Windows PowerShell ISE
    • Windows Powershell Commands
    • WinRM PowerShell
    • PowerShell Date
    • Powershell Write-Host
    • PowerShell Get-ChildItem
    • PowerShell Sort-Object
    • PowerShell Where Object
    • PowerShell Set-Content
    • PowerShell Set-Location
    • PowerShell Invoke-Command
    • PowerShell Invoke-Webrequest
    • PowerShell Get-Location
    • PowerShell Get-Date
    • PowerShell Get-Service
    • PowerShell Test-Path
    • Powershell Module Path
    • PowerShell Out-File
    • PowerShell if File Exists
    • Powershell Copy File
    • PowerShell Delete File
    • PowerShell New-Item
    • PowerShell Rename-Item
    • PowerShell ComputerName
    • PowerShell Get-Content
    • PowerShell Get-Item
    • PowerShell Get-ADUser
    • PowerShell Grep
    • PowerShell Concatenate String
    • PowerShell Get-Process
    • PowerShell Count
    • PowerShell pause
  • Basics
    • PowerShell Restart Service
    • PowerShell comment
    • PowerShell Map Network Drive
    • PowerShell Append to File
    • PowerShell print
    • What is PowerShell
    • Uses Of Powershell
    • PowerShell Empire
    • PowerShell Parameter
    • PowerShell Stop Service
    • PowerShell Versions
    • How To Install PowerShell
    • PowerShell uninstall module
    • How to Use PowerShell?
    • PowerShell Logging
    • PowerShell Tools
    • PowerShell Commands
    • PowerShell Version Command
    • PowerShell Administrator
    • PowerShell Modules
    • PowerShell Registry
    • PowerShell block Comment
    • PowerShell Verbs
    • PowerShell list
    • PowerShell add user to group
    • PowerShell Write to Console
    • Variable in PowerShell
    • PowerShell New Line
    • PowerShell prompt for input
    • PowerShell File Extension
    • Powershell Remotesigned
    • PowerShell Write to File
    • PowerShell Ping
    • PowerShell wget
    • PowerShell Global variable
    • PowerShell Get-ADGroup
    • Array in PowerShell
    • PowerShell Multidimensional Array
    • PowerShell Array of Strings
    • PowerShell? join array
    • Useful PowerShell Scripts
    • String in PowerShell
    • PowerShell Switch Statement
    • PowerShell Function Parameters
    • PowerShell vs PowerShell ISE
    • PowerShell test-connection
    • PowerShell Test-NetConnection
    • PowerShell GUI
    • PowerShell Variable in String
    • PowerShell Active Directory
  • Variables
    • PowerShell Variables
    • PowerShell Environment Variables
    • PowerShell set environment variable
    • Hashtable in PowerShell
    • Set Variable in PowerShell
  • Operators
    • PowerShell Operators
    • Comparison Operators in PowerShell
    • Logical Operators in PowerShell
    • PowerShell Boolean
    • PowerShell Like Operator
  • Control Statements
    • If Statement in PowerShell
    • If Else in PowerShell
    • Else If in PowerShell
    • Loops in PowerShell
    • For loop in PowerShell
    • PowerShell While Loop
    • PowerShell do while
    • PowerShell Loop through Array
    • PowerShell add to array
    • PowerShell ForEach Loop
    • PowerShell Break
    • PowerShell Continue
    • Switch Case in PowerShell
    • PowerShell If-Not
    • Try-catch in PowerShell
  • Functions
    • PowerShell Functions
    • PowerShell String Functions
    • powershell nslookup
    • PowerShell here string
    • PowerShell Wildcards
    • Regex in PowerShell
    • PowerShell not like
    • PowerShell Filter
    • PowerShell Sleep
    • PowerShell where
    • PowerShell join string
    • PowerShell Exit
    • PowerShell null
    • PowerShell Dictionary
    • PowerShell Location
    • PowerShell Start-Service
    • PowerShell is not digitally signed
    • PowerShell Uptime
    • PowerShell Create Directory
    • PowerShell Trim
    • PowerShell Join-Path
    • PowerShell Execution Policy
    • PowerShell SubString
    • PowerShell Format Table
    • PowerShell Import Module
    • PowerShell ForEach Object
    • PowerShell Alias
    • PowerShell Scheduled Task
    • PowerShell Convert String to Date
    • PowerShell Split String
    • PowerShell Multiline String
    • PowerShell MultiLine Comment
    • PowerShell Rename Folder
    • PowerShell Delete Folder
    • PowerShell String Replace
    • PowerShell join
    • PowerShell xcopy
    • PowerShell Base64
    • PowerShell Tail
    • PowerShell User List
    • PowerShell remove User from group
    • PowerShell JSON Format
    • PowerShell Send Mail
    • PowerShell Convert to String
    • PowerShell Start-Process
    • PowerShell change directory
    • PowerShell Open File
    • PowerShell Batch File
    • PowerShell ZIP
    • PowerShell unzip
    • PowerShell XML
    • PowerShell XML Parsing
    • Remote PowerShell
    • PowerShell Escape Character
    • PowerShell scriptblock
    • PowerShell Executable Location
    • PowerShell Import-CSV?
    • PowerShell Export CSV
  • Interview Questions
    • PowerShell Interview Questions

PowerShell Invoke-Webrequest

By Chirag NagarekarChirag Nagarekar

PowerShell Invoke-Webrequest

Definition of PowerShell Invoke-Webrequest

In PowerShell Invoke-WebRequest cmdlet which is a part of Microsoft.PowerShell.Utility Module is one of the Web scrapping methods (the other method is Invoke-RestMethod and in cmd and other OS it is known as CURL) which sends HTTP and HTTPS request to the requested webpage or URI which is called Endpoint and retrieves, deletes, updates content in the form of Headers, Images, Links and other HTML significant elements and for that it uses the multiple methods like GET, POST, PUT, DELETE, etc to send the request, modify the content or to delete the content of the webpage. Invoke-WebRequest is also known to work with REST API.

Syntax:

Start Your Free Data Science Course

Hadoop, Data Science, Statistics & others

Invoke-WebRequest

[-UseBasicParsing] [-Uri] <Uri>
[-WebSession <WebRequestSession>] [-SessionVariable <String>] [-AllowUnencryptedAuthentication] [-Authentication <WebAuthenticationType>] [-Credential <PSCredential>] [-UseDefaultCredentials] [-CertificateThumbprint <String>] [-Certificate <X509Certificate>] [-SkipCertificateCheck] [-SslProtocol <WebSslProtocol>] [-Token <SecureString>] [-UserAgent <String>] [-DisableKeepAlive] [-TimeoutSec <Int32>] [-Headers <IDictionary>] [-MaximumRedirection <Int32>] [-MaximumRetryCount <Int32>] [-RetryIntervalSec <Int32>] [-Method <WebRequestMethod>] [-Proxy <Uri>] [-NoProxy] [-CustomMethod <String>] [-ProxyCredential <PSCredential>] [-ProxyUseDefaultCredentials] [-Body <Object>] [-Form <IDictionary>] [-ContentType <String>] [-TransferEncoding <String>] [-InFile <String>] [-OutFile <String>] [-PassThru] [-Resume] [-SkipHttpErrorCheck] [-PreserveAuthorizationOnRedirect] [-SkipHeaderValidation] [<CommonParameters>]

How does the Invoke-WebRequest method work in PowerShell?

From PowerShell v3.0 onwards PowerShell has added cmdlet Invoke-WebRequest which ships with Microsoft.PowerShell.Utility module to work with WebPages. We can retrieve, remove or update the content of the webpages using PowerShell, apparently this will not be the same experience as page navigation using GUI but it is to automate the web requests and get rid of tedious browsing and manual daily tasks.

Invoke-WebRequest sends a request to the URI (Uniform Resource Identifier) which is also called Endpoint and retrieves the data from the Web Page. It directly works with the URL or with the REST API because some websites allow modifying the content using only APIs.

When we work with the APIs, the output data is generally in JSON or XML format for example,

https://yts.mx/api/v2/list_movies.json

or

https://yts.mx/api/v2/list_movies.xml

In the above case, once the content is retrieved we can convert the content to JSON or XML method using PowerShell.

Examples

Let us discuss examples of PowerShell Invoke-Webrequest.

a.Invoke-WebRequest to retrieve the data from the Webpage.

We can provide the URI to retrieve the content from the webpage. There is a Content and RawContent Property, you can use those properties to retrieve the data. It depends on the webpage, what kind of data we are retrieving. Data can be in HTML format or can be in JSON or XML format.

$req = Invoke-WebRequest -Uri https://theautomationcode.com/feed/
$req

PowerShell Invoke-Webrequest 1\

PowerShell Invoke-Webrequest 2

PowerShell Invoke-Webrequest 3

Once you run the above command you can see the different Properties like StatusCode, StatusDescription, Content, RawContent, Forms, Headers, Images, InputFields, Links, ParsedHtml, RawContentLength, etc.

You can also check the properties of this command using Get-Member command.

$req | gm -MemberType Properties

Output:

PowerShell Invoke-Webrequest 5

To retrieve the content,

$req = Invoke-WebRequest -Uri "https://theautomationcode.com/feed/"
$req.Content

b.Invoke-WebRequest to Check the Website Status code.

When a client sends the request to the endpoint, it sends the handshake status with the various status codes which shows the website availability or any error on the webpage.

You can find various codes with their status message below the link.

https://en.wikipedia.org/wiki/List_of_HTTP_status_codes

To retrieve the status messages using Invoke-WebRequest we need to use the StatusCode as a reference property. For example,

try{
$uri = "http://theautomationcode.com"
$response = Invoke-WebRequest -Uri $uri -Method Get -ErrorAction Stop
Write-Output "Status Code : $($response.statuscode)"
}
catch{
Write-Output "Status Code : $($_.Exception.Response.StatusCode.Value__)"
}

Output:

status code

If we mention a website that does not exist, it will give a different status code. For the below URI,

$uri = "http://theautomationcode.com/failed"

Output:

status code 1

c. Invoke-WebRequest to retrieve images and Links from the Webpage.

We can retrieve images and Link from the webpage using Invoke-WebRequest command. To retrieve images from the webpage and store them to the specific location us the below command,

try{
$wc = New-Object System.Net.WebClient
$uri = "http://theautomationcode.com"
$imgs = (Invoke-WebRequest -Uri $uri -ErrorAction Stop).Images
$count = 0
foreach($imgurl in $imgs.src){
$wc.DownloadFile($imgurl,"C:\temp\WebImages\img$Count.jpg")
$count++
}
}
catch{
Write-Output "Status Code : $($_.Exception.Response.StatusCode.Value__)"
}

Output:

JPG File

Invoke-WebRequest to get links from the WebPage.

$uri = "http://theautomationcode.com"
$response = Invoke-WebRequest -Uri $uri
$response.Links.href

Output:

Invoke-WebRequest

d.Working With Forms in Invoke-Webrequest.

Forms are the pages that stores the information about Input items such as username and Password. We can use forms to submit the username and the password using the POST method.

For example, We will use the Reddit login page and submit credentials there. To retrieve the login page we have URL below for Reddit.

$Response = Invoke-WebRequest https://www.reddit.com/login/ -SessionVariable sv
$Response.Forms

Here we have used Forms Property to retrieve our needed information.

Forms Property

Once you run the above command, you can see the POST method there and we will use it.

$Response = Invoke-WebRequest https://www.reddit.com/login/ -SessionVariable sv
$form = $Response.Forms[2] $form.Fields['loginUsername'] = 'reddit'
$form.Fields['loginPassword'] = 'redditpass'
Invoke-WebRequest $Response -Body $form -WebSession $sv -Method Post

The above script is an example of the Web Forms and POST method.

e. Invoke-WebRequest to get the feeds from the webpage.

Sometimes we just need to retrieve the content of the webpage. For example, Feeds from the webpage might be in the JSON or XML format. In the below example, we will retrieve Reddit RSS feeds from the URL https://www.reddit.com/.rss then later convert them into XML or JSON format to retrieve the feeds in the proper format in the console. For example,

$response = Invoke-WebRequest "https://www.reddit.com/.rss" -Method Get
$response.Content

Once we run the above command, it retrieves the XML output as shown below.

XML output

$response = Invoke-WebRequest "https://www.reddit.com/.rss" -Method Get
$feeds = [XML]$response.Content | Select -ExpandProperty Feed
$feeds.Entry | Select Title, Updated

Output:

title

Conclusion

Invoke-Webrequest is a very useful command to automate web requests and for working with the APIs. There are many free APIs like weather API, movies database, finance APIs that allow integrating with the web request, and in that case, we can retrieve, update, post, or delete the data from the requested APIs.

Recommended Articles

This is a guide to PowerShell Invoke-Webrequest. Here we discuss Definition, How does the Invoke-WebRequest method work in PowerShell? and examples with code implementation. You may also have a look at the following articles to learn more –

  1. PowerShell Execution Policy
  2. PowerShell Sleep
  3. PowerShell SubString
  4. PowerShell Escape Character
Popular Course in this category
PowerShell Training (2 Courses, 1 Project)
  2 Online Courses |  1 Hands-on Project |   4+ Hours |  Verifiable Certificate of Completion
4.5
Price

View Course

Related Courses

Shell Scripting Training (4 Courses, 1 Project)4.9
All in One Data Science Bundle (360+ Courses, 50+ projects)4.8
Data Visualization Training (15 Courses, 5+ Projects)4.7
Primary Sidebar
Footer
About Us
  • Blog
  • Who is EDUCBA?
  • Sign Up
  • Live Classes
  • Corporate Training
  • Certificate from Top Institutions
  • Contact Us
  • Verifiable Certificate
  • Reviews
  • Terms and Conditions
  • Privacy Policy
  •  
Apps
  • iPhone & iPad
  • Android
Resources
  • Free Courses
  • Database Management
  • Machine Learning
  • All Tutorials
Certification Courses
  • All Courses
  • Data Science Course - All in One Bundle
  • Machine Learning Course
  • Hadoop Certification Training
  • Cloud Computing Training Course
  • R Programming Course
  • AWS Training Course
  • SAS Training Course

ISO 10004:2018 & ISO 9001:2015 Certified

© 2023 - EDUCBA. ALL RIGHTS RESERVED. THE CERTIFICATION NAMES ARE THE TRADEMARKS OF THEIR RESPECTIVE OWNERS.

EDUCBA
Free Data Science Course

Hadoop, Data Science, Statistics & others

By continuing above step, you agree to our Terms of Use and Privacy Policy.
*Please provide your correct email id. Login details for this Free course will be emailed to you
EDUCBA

*Please provide your correct email id. Login details for this Free course will be emailed to you

Let’s Get Started

By signing up, you agree to our Terms of Use and Privacy Policy.

EDUCBA

*Please provide your correct email id. Login details for this Free course will be emailed to you
EDUCBA

*Please provide your correct email id. Login details for this Free course will be emailed to you
EDUCBA Login

Forgot Password?

By signing up, you agree to our Terms of Use and Privacy Policy.

This website or its third-party tools use cookies, which are necessary to its functioning and required to achieve the purposes illustrated in the cookie policy. By closing this banner, scrolling this page, clicking a link or continuing to browse otherwise, you agree to our Privacy Policy

Loading . . .
Quiz
Question:

Answer:

Quiz Result
Total QuestionsCorrect AnswersWrong AnswersPercentage

Explore 1000+ varieties of Mock tests View more