Parse HTML in Java with jsoup
When a site offers no API, the HTML itself becomes the data source. jsoup is a Java library that implements the WHATWG HTML5 specification and turns markup into a Document you can query with familiar CSS selectors. This guide walks through installing jsoup with Maven, fetching a page, extracting blog titles, links, and images, and looping over paginated results. It also flags where jsoup stops — pages that depend on JavaScript execution.
Set up a Maven project with jsoup
This walkthrough uses Maven as the build system, but jsoup can be used without it; the jsoup download page covers that path. Generate a Maven project with the archetype command in the supplied setup module, then open the generated pom.xml and add jsoup as a dependency. The example pins version 1.14.3, which was the latest release when the original tutorial was written. Check the jsoup download page for the current version before you build. The Exec Maven Plugin is added so the application can be launched with mvn exec:java. The exact commands and XML snippets are preserved in the setup modules referenced below.
Inspect the target page before writing code
Start with the browser, not the editor. Open the target page, launch developer tools, and use the element picker to see which markup produces each visible item. In the example, the blog listing page contains a div with id content, and each blog card is a div carrying the class p-[10px]. The blog title sits in an h4 element, the link is an a tag, and each card contains two images — a header image and an author avatar. Knowing this structure first prevents selector guesswork later.
Parse HTML into a Document
jsoup converts HTML into a Document object, a programmatic representation of the DOM. The parse method has several overloads: parse(File file, String charsetName) for files (including gzipped ones), parse(InputStream in, String charsetName, String baseUri) for streams, and parse(String html) for a raw string. All return a Document. A quick sanity check is to parse a small HTML string and call document.title(). To fetch a live page, you can use HttpURLConnection and pass the response stream to parse, but that requires boilerplate. jsoup's connect method is shorter: Jsoup.connect(url) returns a Connection you can configure with data, header, cookie, and timeout, then finish with get() or post() to receive the parsed Document. Wrap the call in a try-catch because get() can throw IOException.
Add the documented org.jsoup:jsoup dependency to your Maven project. Save this standalone class as ParseDocument.java; it parses a local HTML string without fetching a website.
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
public class ParseDocument {
public static void main(String[] args) {
String html = "<html><head><title>Product catalogue</title></head><body></body></html>";
Document page = Jsoup.parse(html);
System.out.println(page.title());
}
}Choose the right selector method
jsoup offers several ways to locate elements, and the return type matters. Use this checklist to pick the right one:
- getElementById(id) returns a single Element for a unique id. In the example, document.getElementById("content") returns the content div, and childrenSize() reports 2 children.
- getElementsByClass(className) returns an Elements collection because multiple elements can share a class. document.getElementsByClass("p-[10px]") returns the blog cards, which you can iterate with a for loop and read via text().
- select(cssSelector) returns an Elements list matching a CSS selector, similar to querySelectorAll. blog.select("h4").text() pulls the title text.
- attr(name) returns the named attribute of the first matching element. blog.select("a").attr("href") yields the link.
- first() returns the first element in an Elements list; selectFirst(selector) does the same in one call. Because each card has two images, blog.selectFirst("img").attr("src") reliably targets the header image rather than the avatar.
- Attribute selectors narrow by attribute content. img[src*=authors] matches images whose src contains "authors", which isolates the author avatar.
If you know JavaScript's DOM methods, these names will feel familiar. The key distinction is Element versus Elements: single-result methods return one Element, while class and selector queries return a collection you must iterate or reduce with first().
Save as BlogListing.java with jsoup available and set SITE_BASE_URL. This assumes the documented example markup: p-[10px] entry classes, h4 titles and anchor/image descendants. Selectors are specific to that HTML, and a missing cover image is handled explicitly.
import java.io.IOException;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
public class BlogListing {
public static void main(String[] args) throws IOException {
String origin = System.getenv("SITE_BASE_URL");
if (origin == null) throw new IllegalArgumentException("Set SITE_BASE_URL");
Document page = Jsoup.connect(origin.replaceAll("/+$", "") + "/blog/")
.timeout(5000).get();
for (Element entry : page.getElementsByClass("p-[10px]")) {
String title = entry.select("h4").text();
String link = entry.select("a").attr("href");
Element cover = entry.selectFirst("img");
String image = cover == null ? "" : cover.attr("src");
String authorImage = entry.select("img[src*=authors]").attr("src");
System.out.println(title);
System.out.println(link);
System.out.println(image);
System.out.println(authorImage);
}
}
}Handle pagination with a URL pattern
The blog listing uses a predictable URL scheme: page one lives at /blog/, and later pages at /blog/page/N/. A for loop from 1 to 4 can build each URL, fetch it with Jsoup.connect(url).timeout(5000).get(), and run the same extraction logic. The first page is the exception — it has no /page/1/ suffix — so the loop uses a conditional to pick the base URL for i == 1. The final code module combines the pagination loop with the selector calls and prints titles, links, header images, and author images for each page.
Save as BlogPages.java with jsoup available. This keeps the documented four-page example, 5000 ms timeout, /blog/ first route and /blog/page/N subsequent routes. Set SITE_BASE_URL to a matching test site; adjust selectors only after inspecting its HTML.
import java.io.IOException;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
public class BlogPages {
public static void main(String[] args) {
String origin = System.getenv("SITE_BASE_URL");
if (origin == null) throw new IllegalArgumentException("Set SITE_BASE_URL");
origin = origin.replaceAll("/+$", "");
for (int number = 1; number <= 4; number++) {
String address = number == 1 ? origin + "/blog/" : origin + "/blog/page/" + number;
try {
Document page = Jsoup.connect(address).timeout(5000).get();
for (Element entry : page.getElementsByClass("p-[10px]")) {
System.out.println(entry.select("h4").text());
System.out.println(entry.select("a").attr("href"));
Element cover = entry.selectFirst("img");
System.out.println(cover == null ? "" : cover.attr("src"));
System.out.println(entry.select("img[src*=authors]").attr("src"));
}
} catch (IOException error) {
System.err.println(address + ": " + error.getMessage());
}
}
}
}Know when jsoup is not enough
jsoup is efficient and easy to use for static HTML, but it does not execute JavaScript. Pages that render content client-side — single-page applications and JavaScript-heavy sites — will return incomplete or empty markup to jsoup. In those cases you need a browser automation tool such as Selenium, or a service that renders pages before returning HTML. The jsoup documentation covers additional APIs beyond this walkthrough.
Frequently asked questions
Which jsoup version should I use?
The example pins 1.14.3, which was the latest release when the tutorial was written. Check the jsoup download page for the current version and update the dependency in pom.xml accordingly.
Can jsoup scrape pages that require JavaScript?
No. jsoup parses the HTML it receives but does not run JavaScript, so content injected by client-side scripts will not appear in the Document. Use a browser automation tool or a rendering service for those pages.
What is the difference between Element and Elements in jsoup?
Element represents a single node, returned by methods like getElementById and selectFirst. Elements is a collection returned by getElementsByClass and select, because multiple elements can match. Iterate over Elements or call first() to reduce it to one Element.
How do I get an attribute value such as a link or image source?
Select the element first, then call attr with the attribute name. For example, blog.select("a").attr("href") returns the link, and blog.selectFirst("img").attr("src") returns the first image source.