Jsoup error 403 Forbidden exception fix example shows how to fix error 403 – Forbidden exception while using Jsoup in Java. Jsoup throws “org.jsoup.HttpStatusException: HTTP error fetching URL. Status=403” exception.
How to solve error 403 – forbidden exception while using the Jsoup?
When trying to parse a website, I encountered the “org.jsoup.HttpStatusException: HTTP error fetching URL. Status=403” exception. Here is the code which I used to connect to the website. I havev changed the website name.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
package com.javacodeexamples.libraries.jsoup; import java.io.IOException; import org.jsoup.Connection; import org.jsoup.Jsoup; import org.jsoup.nodes.Document; public class Jsoup403ForbiddenExample { public static void main(String[] args) { try{ //connect to the website Connection connection = Jsoup.connect("http://www.example.com"); //get the HTML document Document doc = connection.get(); //parse text from HTML String strHTML = doc.text(); }catch(IOException ioe){ ioe.printStackTrace(); } } } |
Output
|
1 2 3 4 5 6 |
org.jsoup.HttpStatusException: HTTP error fetching URL. Status=403, URL=http://www.example.com at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:537) at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:493) at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:205) at org.jsoup.helper.HttpConnection.get(HttpConnection.java:194) at com.javacodeexamples.libraries.jsoup.Jsoup403ForbiddenExample.main(Jsoup403ForbiddenExample.java:19) |
The HTTP status (response code) returned was 403.
What is HTTP 403 status code?
From Wikipedia,
A web server may return a 403 Forbidden HTTP status code in response to a request from a client for a web page or resource to indicate that the server can be reached and understood the request, but refuses to take any further action.
How to resolve the error 403 while using Jsoup?
Many of the websites filter web requests against the list of known user agents and deny access to unknown or bot user agents. Jsoup uses the Java version as a user agent string while making the request. This falls into the later category of bots and that could be the reason to get the 403 error – access is forbidden.
A simple solution to this problem is to specify the user agent string while connecting to the website using Jsoup as given below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
try{ //connect to the website Connection connection = Jsoup.connect("http://whatsmyuseragent.com"); //specify user agent connection.userAgent("Mozilla/5.0"); //get the HTML document Document doc = connection.get(); //parse text from HTML String strHTML = doc.text(); }catch(IOException ioe){ ioe.printStackTrace(); } |
Are you still getting Jsoup 403 errors? Other possible causes of getting 403 error – access is forbidden are given below.
1) Specify the HTTP referrer header while making a request using Jsoup.
2) If you are crawling the web too fast, the website’s firewall may block your IP. Wait for a few milliseconds before making the next request using the Thread.sleep method.
If you are already using the user agent value, try changing the user agent value that reflects the newer version of the browser. Many times this solves the error.
This example is a part of the Jsoup tutorial with examples.
Please let me know your views in the comments section below.
About the author
- Gson – Serialize HashMap null entries - April 5, 2023
- Gson: Convert String to JsonObject without POJO - April 5, 2023
- Gson – Convert JSON to a Typed ArrayList - April 5, 2023
I can’t express my gratitude to you. Thank You so much. I had been looking for a solution for 2 days continuously and finally found one.