Call the HTML API from Go: a first-request walkthrough
This guide walks through the smallest useful Go client for the HTML API.
You will construct a GET request, escape the target URL, attach your API key, and then decide whether to print the response body or write it to a local HTML file. The two public code modules below are the exact contracts to preserve; the surrounding prose explains how to read and adapt them.
What the request is made of
Every call in this walkthrough follows the same shape: a standard-library HTTP client, a target URL encoded as a query-parameter value, and a GET request to the HTML API endpoint carrying both the API key and the escaped URL as query parameters. The response object is then handed back to the caller. Before you write any code, map the values you control against the values the program derives:
- You supply: the API origin, API key and target URL.
- The program derives: the escaped URL, the http.Request, and the *http.Response.
- You decide: whether the response body is printed to stdout or written to a file.
Why escaping matters here: the target URL is being placed inside another URL's query string, not used on its own. A raw target like https://example.com/search?q=go&page=2 already contains ? and &, which the outer request would otherwise read as its own parameter separators. url.QueryEscape, or the url.Values.Encode helper used below, encodes the query value so the whole target survives as a single parameter. Without encoding, a target containing query delimiters can be split into unintended parameters. That does not guarantee a construction error: a request may still be sent with the wrong target value. Keeping the supply/derive/decide split clear also makes it easier to spot mistakes: missing local configuration can be detected before sending, while target validity and an unsuccessful HTTP status may require inspecting the service response.
The reusable request function
The reusable request function is the shared contract behind both public modules. It creates an http.Client, escapes the target URL with url.QueryEscape, builds a GET request against the HTML API endpoint with the API key and escaped URL as query parameters, calls ParseForm, and executes the request with client.Do. On a transport error it prints a failure message and still returns the response value. Treat this contract as fixed: keep the endpoint path, parameter names, and function signature as supplied. The file-saving module below shows this function with the API key and target URL lifted into parameters, so the same function can be called with a different page each time.
Set APP_BASE_URL, API_KEY and TARGET_URL before running this Go program. It preserves GET /api/v1/ with api_key and url query fields. URL encoding, response-body closure and both request/read errors are handled explicitly.
package main
import (
"fmt"
"io"
"net/http"
"net/url"
"os"
"strings"
)
func fetchHTML(target string) (*http.Response, error) {
origin := strings.TrimRight(os.Getenv("APP_BASE_URL"), "/")
key := os.Getenv("API_KEY")
if origin == "" || key == "" {
return nil, fmt.Errorf("set APP_BASE_URL and API_KEY")
}
query := url.Values{"api_key": {key}, "url": {target}}
request, err := http.NewRequest(http.MethodGet, origin+"/api/v1/?"+query.Encode(), nil)
if err != nil { return nil, err }
return http.DefaultClient.Do(request)
}
func main() {
target := os.Getenv("TARGET_URL")
if target == "" { fmt.Fprintln(os.Stderr, "set TARGET_URL"); return }
response, err := fetchHTML(target)
if err != nil { fmt.Fprintln(os.Stderr, err); return }
defer response.Body.Close()
body, err := io.ReadAll(response.Body)
if err != nil { fmt.Fprintln(os.Stderr, err); return }
fmt.Println(response.Status)
fmt.Println(response.Header)
fmt.Println(string(body))
}Reading and printing the response
The second module shows how to consume the response returned by the request function: read the body with ioutil.ReadAll, then print the status, headers, and body. This is the quickest way to confirm that your key and URL are accepted before you add file output. As a reader tip rather than a documented API behavior, it is worth printing the status alongside the body: a body that looks empty or unexpected is easier to interpret when you can see the status code next to it.
Saving the HTML to a file
The third module turns the request into a small tool: it accepts an API key and a user URL as parameters, fetches the page, and writes the response body to page.html with os.Create and file.WriteString. The example target is the blog path on the site base URL. Use this module when you need a local copy for later parsing rather than an on-screen dump.
A short decision checklist before you run it:
- Have you replaced the placeholder API key with your own value?
- Is the target URL a full URL, not just a path?
- Do you want the body printed or written to disk?
- If writing to disk, does the working directory allow file creation?
- After the run, did you check the response status before trusting the saved file?
Add this function to the same package as fetchHTML, using its existing io and os imports. Call saveHTML(target, "page.html") from main and check the returned error; do not add a second main function. A successful file write does not itself prove an HTTP success status.
func saveHTML(target, path string) error {
response, err := fetchHTML(target)
if err != nil { return err }
defer response.Body.Close()
body, err := io.ReadAll(response.Body)
if err != nil { return err }
return os.WriteFile(path, body, 0600)
}Distinguishing the two failure points
The two initial error boundaries are request construction and network transport. The helper checks that the API origin and key are present before building the request. http.NewRequest can reject a malformed request URL; it does not validate the API key or the page named by the url query value. No ParseForm call is needed for this encoded GET request. Next, http.Client.Do can return a transport error when the round trip cannot be completed. An HTTP response with an unsuccessful status is different: inspect its status and body before changing the request. After a response arrives, io.ReadAll and a local file write can also fail, so the examples return or report those output errors separately.
Frequently asked questions
Why is the target URL escaped before the request?
The target URL is passed as a query parameter, so reserved characters inside it would otherwise break the request line. url.QueryEscape converts the URL into a safe parameter value before it is concatenated into the endpoint string.
What does the program do if client.Do returns an error?
The supplied code prints a failure message and still returns the response value. Callers should therefore check the response before reading its body, because a transport failure may leave the response unusable.
Can I reuse the request function for different pages?
Yes. In the file-saving version, the request function takes the API key and user URL as parameters, so the same function can be called with a different target URL each time. The earlier version hardcodes placeholders and is best treated as a template.