Skip to content

Automate Invoice Downloads with HtmlUnit and Java

This guide walks through a small Java utility that signs in to a billing dashboard, reads the invoice history table, converts each row into a Bill object, prints the object as JSON, and saves the matching PDF invoice to disk. The example uses HtmlUnit for the browser session and Jackson for JSON output.

Sign in and reach the billing page

The first step is to open a WebClient session and submit the login form. The original example reuses an autoLogin helper that fills in the email and password fields without inspecting the DOM. After login, the code requests the billing settings page and checks the page text for a sign-in warning. If the warning appears, the program throws an exception so the failure is visible instead of silently producing empty results.

The protected login module (code0001) contains the exact base URL, login call, billing page URL, and the error message used to detect a failed sign-in. Keep the credentials out of source control; the example uses placeholder strings for email and password.

Model each invoice as a Bill object

A Bill class holds four fields: a label, an amount as BigDecimal, a date, and a URL. The protected module code0002 shows the exact field names and types. Keeping the amount as BigDecimal avoids floating-point rounding when the value is later serialized or compared.

A useful field map for the reader:

  • label — the invoice description text from the second table column, for example "Invoice for December 2015".
  • amount — the third column with the dollar sign removed, parsed as BigDecimal.
  • date — the first column parsed with the pattern "MMMM d, yyyy" and Locale.ENGLISH.
  • url — the href attribute of the anchor in the fourth column.

If your billing page uses a different column order, adjust the getCell indexes rather than the field names.

Read the invoice table with HtmlTable

HtmlUnit exposes HTML tables through the HtmlTable class. The example selects the table by XPath, iterates over the first table body's rows, and reads cells by index. Only rows whose label contains "Invoice" are kept, so payment rows are skipped. Each kept row becomes a Bill, and Jackson's ObjectMapper writes the Bill as a JSON string that is printed to standard output.

The protected module code0003 contains the exact XPath, cell indexes, date format, amount cleanup, and Jackson call. The XPath targets a table with the class "listing Billing--history"; if the dashboard markup changes, update the selector and re-check the column indexes.

Download the PDF and verify the content type

For each invoice URL, the program fetches the page and checks that the response content type is application/pdf before writing the bytes to a file. The output filename combines a prefix, the invoice label, and the .pdf extension. The protected module code0004 contains the exact content-type comparison and the IOUtils.copy call.

A short worked example using one captured row: the December 2015 invoice has label "Invoice for December 2015", amount 0.35, date 1451602800000, and URL /billing/XXXXX.pdf. After the content-type check passes, the file is written as DigitalOceanInvoice for December 2015.pdf. The JSON line printed for that row is shown in the protected console module code0005.

Pre-flight checklist before you run the bot

Use this checklist to separate the three common failure points:

  1. Login failure — the billing page text contains the sign-in warning. Check the email and password placeholders and confirm the login URL.
  2. Table-parse failure — the XPath returns null or the row count is zero. Inspect the current dashboard markup and update the table class or column indexes.
  3. PDF failure — the response content type is not application/pdf. The URL may point to an HTML preview page; log the content type before writing the file.

Each failure has a different fix, so log the stage name alongside the exception. The original example prints the stack trace, which is enough for a local run but not for an unattended job.

Frequently asked questions

Why use HtmlUnit instead of a full browser?

HtmlUnit runs without a visible browser and provides table and form helpers, which suits a small scheduled job. If the billing dashboard relies heavily on JavaScript rendering, a full browser may be needed instead.

Why does the JSON date appear as a large number?

Jackson serializes java.util.Date as epoch milliseconds by default. The value 1451602800000 corresponds to the December 2015 invoice date in the captured output.

What if the invoice URL returns HTML instead of a PDF?

The content-type check fails and the file is not written. Log the returned content type and the URL, then confirm whether the dashboard requires an extra request or a different link for the PDF download.